Video Saliency Detection Using Deep Convolutional Neural Networks
Numerous deep learning based efforts have been done for image saliency detection, and thus, it is a natural idea that we can construct video saliency model on basis of these image saliency models in an effective way. Besides, as for the limited number of training videos, existing video saliency model is trained with large-scale synthetic video data. In this paper, we construct video saliency model based on existing image saliency model and perform training on the limited video data. Concretely, our video saliency model consists of three steps including feature extraction, feature aggregation and spatial refinement. Firstly, the concatenation of current frame and its optical flow image is fed into the feature extraction network, yielding feature maps. Then, a tensor, which consists of the generated feature maps and the original information including the current frame and the optical flow image, is passed to the aggregation network, in which the original information can provide complementary information for aggregation. Finally, in order to obtain a high-quality saliency map with well-defined boundaries, the output of aggregation network and the current frame are used to perform spatial refinement, yielding the final saliency map for the current frame. The extensive qualitative and quantitative experiments on two challenging video datasets show that the proposed model consistently outperforms the state-of-the-art saliency models for detecting salient objects in videos.
Video saliency Convolutional neural networks Feature aggregation
Xiaofei Zhou Zhi Liu Chen Gong Gongyang Li Mengke Huang
Institute of Information and Control,Hangzhou Dianzi University,Hangzhou,China;Shanghai Institute fo Shanghai Institute for Advanced Communication and Data Science,Shanghai University,Shanghai,China;Sc Key Laboratory of Intelligent Perception and Systems for High-Dimensional Information of Ministry of
国际会议
广州
英文
308-319
2018-11-23（万方平台首次上网日期，不代表论文的发表时间）