New Word Detection Algorithm for Chinese Based on Extraction of Local Context Information

摘要：

Chinese segmentation is an important issue inChinese text processing.The traditional segmentationmethods those depend on an existing dictionary sufferthe drawbacks when encounter unknown words.Thepaper proposed a segmenting algorithm for Chinesebased on extracting local context information.It addedthe context information of the testing text into the localPPM statistical model so as to guide the detection ofnew words.The algorithm focusing on the process ofonline segmentation and new word detection achievesa good effect in the close or opening test,andoutperforms some well-known Chinese segmentationsystem to a certain extent.

作者: Hua-Lin Zeng Chang-Le Zhou Xiao-Dong Shi Tang-Qiu Li Chang Su

作者单位: Department of Cognitive Science,Xiamen University,Xiamen 361005,China

会议类型: 国际会议

会议名称: 2008 3rd International Conference on Intelligent System and Knowledge Engineering(第三届智能系统与知识工程国际会议)(ISKE 2008)

会议地点: 厦门

会议语种:英文

页码: 797-801

在线出版日期: 2008-11-17（万方平台首次上网日期，不代表论文的发表时间）