A Normalized Encoder-Decoder Model for Abstractive Summarization Using Focal Loss

摘要：

　　Abstractive summarization based on seq2seq model is a popular research topic today.And pre-trained word embedding is a common unsupervised method to improve deep learning models performance in NLP.However,during applying this method directly to the seq2seq model,we find it does not achieve the same good result as other fields because of an over training problem.In this paper,we propose a normalized encoder-decoder structure to address it,which can prevent the semantic structure of pre-trained word embedding from being destroyed during training.Moreover,we use a novel focal loss function to help our model focus on those examples with low score for getting better performance.We conduct the experiments on NLPCC2018 share task 3: single document summary.Result showed that these two mechanisms are extremely useful,helping our model achieve state-of-the-art ROUGE scores and get the first place in this task from the current rankings.

关键词： Summarization Seq2Seq Pre-trained word embedding Normalized encoder-decoder structure Focal loss

作者: Yunsheng Shi Jun Meng Jian Wang Hongfei Lin Yumeng Li

作者单位: Dalian University of Technology,Dalian 116023,Liaoning,China

会议类型: 国际会议

会议名称: 2018自然语言处理与中文计算国际会议(NLPCC2018)

会议地点: 呼和浩特

会议语种:英文

页码: 383-392

在线出版日期: 2018-08-26（万方平台首次上网日期，不代表论文的发表时间）