Multi-modal Gesture Recognition using Integrated Model of Motion, Audio and Video
Gesture recognition is needed in many practical applications such as human-robot interaction and sign language recognition.In this paper,we propose the novel model that integrates multi-modal features of motion,audio and video captured from Kinect.The proposed framework is able to recognize complex motion by using these modal features.We use Hidden Markov Models or Random Forests to construct motion and audio classifiers or video classifiers.In motion and audio classifiers,we choose feature representation suitable for gesture recognition by comparing multiple features and trained models.To test the availability of the proposed framework,we also compare the performance of the unimodal models and the integrated multi-modal models.In the experiments,we use dataset provided by MMGRC,which is a workshop for Multi-Modal Gesture Recognition Challenge,and the result shows that the proposed framework scored the best correct recognition rate.This means that the modals complement each other and their combination leads to the improvement of gesture recognition.
Gesture Recognition Multi-modal Integration Hidden Markov Model Random Forests
Yusuke Goutsu Takaki Kobayashi Junya Obara Ikuo Kusajima Kazunari Takeichi Wataru Takano Yoshihiko Nakamura
Mechano-Informatics,The University of Tokyo,7-3-1 Hongo,Bunkyo-ku,Tokyo,Japan
国际会议
天津
英文
1-7
2014-07-06（万方平台首次上网日期，不代表论文的发表时间）