Maximum Entropy Modeling with Feature Selection for Text Categorization

摘要：

Maximum entropy provides a reasonable way of estimating probability distributions and has been widely used for a number of language processing tasks.In this paper,we explore the use of different feature selection methods for text categorization using maximum entropy modeling.We also propose a new feature selection method based on the difference between the relative document frequencies of a feature for both relevant and irrelevant classes.Our experiments on the Reuters RCV1 data set show that our own feature selection performs better than the other feature selection methods and maximum entropy modeling is a competitive method for text categorization.

关键词： Text Categorization Feature Selection Maximum Entropy Modeling

作者: Jihong Cai Fei Song

作者单位: Department of Computing and Information Science University of Guelph,Guelph,Ontario,Canada NIG 2W1

会议类型: 国际会议

会议名称: 4th Asia Information Retrieval Symposium(AIRS 2008)(第四届亚洲信息检索研讨会)

会议地点: 哈尔滨

会议语种:英文

页码: 549-554

在线出版日期: 2008-01-16（万方平台首次上网日期，不代表论文的发表时间）