Sentence Alignment Method Based on Maximum Entropy Model Using Anchor Sentences

摘要：

　　The paper proposes a sentence alignment method based on maximum entropy model using anchor sentences to align ancient and modern Chinese sentences in historical classics.The method selects the sentence pairs with the same phrases at the beginning or the end of the sentence or with the same time phrases as anchor sentence pairs,which are employed to divide the paragraph into several sections.Then,the sentences in each section are aligned using dynamic programming algorithm according to the entropy calculated by maximum entropy model.The maximum entropy model employs improved Chinese co-occurrence character feature,length feature and sentence alignment mode feature.The Chinese co-occurrence characters feature is improved by giving different weights to characters in different position based on the contribution to align sentences.In the experiment performed on ShiJi,the precision and recall of the proposed method reaches 95.9%and 95.6%respectively,which outperforms other sentence alignment methods significantly.

关键词： anchor sentences maximum entropy model Chinese co-occurrence character sentence alignment

作者: Chao Che Wenwen Guo Jianxin Zhang

作者单位: Key Laboratory of Advanced Design and Intelligent Computing(Dalian University),Ministry of Education,Dalian

会议类型: 国内会议

会议名称: 第十五届全国计算语言学学术会议(CCL2016)暨第四届基于自然标注大数据的自然语言处理国际学术研讨会(NLP-NABD-2016)

会议地点: 烟台

会议语种:英文

页码: 1-11

在线出版日期: 2016-10-14（万方平台首次上网日期，不代表论文的发表时间）