Image Search by Modality Analysis: A Study of Color Semantics
In this paper, we present a novel image search system, color-based image search by modality analysis. This system enables users to search images with modality words as query, like “romantic, “quiet, etc. Modality here is defined as emotions and conscious subjective aspect of feeling (interests, attitudes, opinions, etc). Modality presents richer semantics than emotions on human perception and understanding of images. We first propose seven modality dimensions to quantitatively analyze modality. A Chinese corpus containing 1200 modality words is constructed, in which each word is annotated with seven-dimensional modality vector. Then three-color combinations are extracted from images as visual signatures by utilizing K-means clustering algorithm and CIE L*A*B* color space. To bridge the gap between modality (high-level semantic concepts) and three-color combinations (visual signatures), the prediction model for modality vectors is built up based on Kobayashi’s Color Image Scale by using the decision tree algorithm. Finally, we establish the image search system. Objective evaluations show the accuracy of the prediction model for modality vectors, while subjective evaluation results indicate that the extracted three-color combinations are in agreement with human perception and search results of our system are consistent with queries.
Xiaohui WANG Jia JIA Jiaming YIN Yongxin WANG Lianhong CAI
Key Laboratory of Pervasive Computing, Ministry of Education Tsinghua National Laboratory for Inform Key Laboratory of Pervasive Computing, Ministry of Education Tsinghua National Laboratory for Inform
国际会议
2011亚太信号与信息处理协会年度峰会(APSIPAASC 2011)
西安
英文
1-9
2011-10-18（万方平台首次上网日期，不代表论文的发表时间）