The Accurate Guidance for Image Caption Generation
Image caption task has been focusing on generating a descriptive sentence for a certain image. In this work, we propose the accurate guidance for image caption generation, which guides the caption model to focus more on the principle semantic object while making human reading sentence, and generate high quality sentence in grammar. In particular, we replace the classification network with object detection network as the multi-level feature extracter to emphasize what human care about and avoid unnecessary model additions. Attention mechanism is utilized to align the feature of principle objects with words in the semantic sentence. Under these circumstances, we combine the object detection network and the text generation model together and it becomes an end-to-end model with less parameters. The experimental results on MSCOCO dataset show that our methods are on part with or even outperforms the current state-of-the-art.
Image caption Object detection Attention mechanism Deep learning
Xinyuan Qi Zhiguo Cao Yang Xiao Jian Wang Chao Zhang
National Key Lab of Science and Technology of Multispetral Information Processing,School of Automation,Huazhong University of Science and Technology,Wuhan 430074,Hubei,China
国际会议
广州
英文
15-26
2018-11-23（万方平台首次上网日期，不代表论文的发表时间）