Improved Viterbi Algorithm-Based HMM2 for Chinese Words Segmentation

摘要：

In order to solve problems caused by the individualism of Chinese architecture more and more researchers focus on Hybrid and improved Hidden Markov Model. However, as the foundation of Chinese natural language processing, studies on Chinese words segmentation based on Secondorder Hidden Markov Model (HMM2) are not abundant. A words frequency weighted smoothing method and a Threshold-Viterbi algorithm are proposed and combined to build a Improved Viterbi Algorithm-based HHM2(IV-HMM2) model in this article to overcome the sparse problem and improve the accuracy. Experimental rusults demonstrate that the improved model has better performance and lower overhead than traditional HMM2.

关键词： Threshold-Viterbi Algorithm IV-HMM2 Chinese words segmentation

作者: Lei La Qiao Guo Dequan Yang Qimin Cao

作者单位: School of Automation Beijing Institute of Technology Beijing, China

会议类型: 国际会议

会议名称: 2012 International Conference on Computer Science and Electronic Engineering(2012 IEEE计算机科学与电子工程国际会议 ICCSEE 2012)

会议地点: 杭州

会议语种:英文

页码: 266-269

在线出版日期: 2012-03-23（万方平台首次上网日期，不代表论文的发表时间）