Cost-aware Learning Rate for Neural Machine Translation

摘要：

　　Neural Machine Translation(NMT)has drawn much attention due to its promising translation performance in recent years.The conventional optimiza-tion algorithm for NMT sets a unified learning rate for each gold target word dur-ing training.However,words under different probability distributions should be handled differently.Thus,we propose a cost-aware learning rate method,which can produce different learning rates for words with different costs.Specifically,for the gold word which ranks very low or has a big probability gap with the best candidate,the method can produce a larger learning rate and vice versa.The extensive experiments demonstrate the effectiveness of our proposed method.

关键词： Neural Machine Translation Cost-aware Learning Rate

作者: Yang Zhao Yining Wang Jiajun Zhang Chengqing Zong

作者单位: National Laboratory of Pattern Recognition,Institute of Automation,CAS University of Chinese Academy of Sciences

会议类型: 国内会议

会议名称: 第十六届全国计算语言学学术会议暨第五届基于自然标注大数据的自然语言处理国际学术研讨会

会议地点: 南京

会议语种:英文

页码: 1-9

在线出版日期: 2017-10-13（万方平台首次上网日期，不代表论文的发表时间）