Learning from LDA Using Deep Neural Networks

摘要：

　　Bayesian models and neural models have demonstrated their respective advantage in topic modeling.Motivated by the dark knowledge transfer approach proposed by 3,we present a novel method that combines the advantages of the two model families.Particularly,we present a transfer learning method that uses LDA to supervise the training of a deep neural network(DNN),so that the DNN can approximate the LDA inference with less computation.Our experimental results show that by transfer learning,a simple DNN can approximate the topic distribution produced by LDA pretty well,and deliver competitive performance as LDA on document classification,with much faster computation.

作者: Dongxu Zhang Tianyi Luo Dong Wang

作者单位: CSLT,RIIT,Tsinghua University;PRIS,Beijing University of Posts and Telecommunications Beijing,China CSLT,RIIT,Tsinghua University CSLT,RIIT,Tsinghua University;Tsinghua National Lab for Information Science and Technology

会议类型: 国际会议

会议名称: 第五届自然语言处理与中文计算会议(NLPCC-ICCPOL2016)

会议地点: 昆明

会议语种:英文

页码: 1-8

在线出版日期: 2016-12-02（万方平台首次上网日期，不代表论文的发表时间）