Instruction-guided Object Detection
This paper aims at instruction-guided object detection,i.e.,predicting the objects associated with the implementation of a specific instruction for intelligent robots.It is not practical to solve this problem by picking out the instruction-related objects from the detection results of a general object detec-tor due to massive annotation cost and low adaptability of training data.To address these challenges,we introduce a flexible dataset that can well adapt to the variation of the instruction set and only annotates instruction-related object samples.We then propose to amend the current detection paradigm by incorporating semantic instruction description effectively.Specifically,the relationship between an instruc-tion and related objects is modeled by the and-or graph and is further fused into a unified neural network for solving the object detection problem constrained by instructions.Our algorithm encodes the and-or graph representation to attend related objects sensitive local features and then infer instruction-related object location.Extensive evaluation on the newly constructed dataset verifies the effectiveness of our approach.
object detection instruction attention modeling and-or graph
Lili Huang Hefeng Wu Guanbin Li Qing Wang
Sun Yat-Sen University Guangzhou,China Guangdong University of Foreign Studies Sun Yat-Sen University Guangzhou,China
国际会议
2019国图灵大会(ACM Turing Celebration conference-China 2019 )
成都
英文
219-224
2019-05-17（万方平台首次上网日期，不代表论文的发表时间）