Non-autoregressive Neural Machine Translation with Distortion Model

摘要：

　　Non-autoregressive translation(NAT)has attracted atten-tion recently due to its high efficiency during inference.Unfortunately,it performs significantly worse than the autoregressive translation(AT)model.We observe that the gap between NAT and AT can be remark-ably narrowed if we provide the inputs of the decoder in the same order as the target sentence.However,existing NAT models still initialize the decoding process by copying source inputs from left to right,and lack an explicit reordering mechanism for decoder inputs.To address this prob-lem,we propose a novel distortion model to enhance the decoder inputs so as to further improve NAT models.The distortion model,incorpo-rated into the NAT model,reorders the decoder inputs to close the word order of the decoder outputs,which can reduce the search space of the non-autoregressive decoder.We verify our approach empirically through a series of experiments on three similar language pairs(En?De,En?Ro,and De?En)and two dissimilar language pairs(Zh?En and En?Ja).Quantitative and qualitative analyses demonstrate the effectiveness and universality of our proposed approach.

关键词： Neural machine translation Non-autoregressive translation Distortion model

作者: Long Zhou Jiajun Zhang Yang Zhao Chengqing Zong

作者单位: National Laboratory of Pattern Recognition,Institute of Automation,CAS,Beijing,People's Republic of National Laboratory of Pattern Recognition,Institute of Automation,CAS,Beijing,People's Republic of

会议类型: 国际会议

会议名称: 9th CCF International Conference on Natural Language Processing and Chinese Computing (NLPCC 2020)

会议地点: 郑州

会议语种:英文

页码: 403-415

在线出版日期: 2020-10-14（万方平台首次上网日期，不代表论文的发表时间）