Multimodal Joint Representation for User Interest Analysis on Content Curation Social Networks
Content curation social networks(CCSNs),where users share interests by images and their text descriptions,are booming social networks.For the purpose of fully utilizing user-generated contents to analysis user interests on CCSNs,we propose a framework of learning multimodal joint representations of pins for user interest analysis.First,images are automatically annotated with category distributions,which benefit from the network characteristics and represent interests of users.Further,image representations are extracted from an intermediate layer of a fine-tuned multilabel convolutional neural network(CNN)and text representations are obtained with a trained Word2Vec.Finally,a multimodal deep Boltzmann machine(DBM)are trained to fuse two modalities.Experiments on a dataset from Huaban demonstrate that using category distributions instead of single categories as labels to fine-tune CNN significantly improve the performance of image representation,and multimodal joint representations perform better than either of unimodal representations.
Multimodal Content curation social networks User modeling Recommender systems
Lifang Wu Dai Zhang Meng Jian Bowen Yang Haiying Liu
Faculty of Information Technology,Beijing University of Technology,Beijing,China
国际会议
广州
英文
363-374
2018-11-23（万方平台首次上网日期，不代表论文的发表时间）