...
  • 主题
高级搜索  >
历史搜索 清空历史
首页> 外文期刊>ACM transactions on Asian and low-resource language information processing >Improving Neural Machine Translation with Linear Interpolation of a Short-Path Unit
【24h】

Improving Neural Machine Translation with Linear Interpolation of a Short-Path Unit

机译：用短路单位的线性插值改善神经机平移

获取原文
获取原文并翻译 | 示例
           
页面导航
  • 摘要
  • 著录项
  • 相似文献
  • 相关主题

摘要

In neural machine translation (NMT), the source and target words are at the two ends of a large deep neural network, normally mediated by a series of non-linear activations. The problem with such consequent non-linear activations is that they significantly decrease the magnitude of the gradient in a deep neural network, and thus gradually loosen the interaction between source words and their translations. As a result, a source word may be incorrectly translated into a target word out of its translational equivalents. In this article, we propose short-path units (SPUs) to strengthen the association of source and target words by allowing information flow over adjacent layers effectively via linear interpolation. In particular, we enrich three critical NMT components with SPUs: (1) an enriched encoding model with SPU, which interpolates source word embeddings linearly into source annotations; (2) an enriched decoding model with SPU, which enables the source context linearly flow to target-side hidden states; and (3) an enriched output model with SPU, which further allows linear interpolation of target-side hidden states into output states. Experimentation on Chinese-to-English, English-to-German, and low-resource Tibetan-to-Chinese translation tasks demonstrates that the linear interpolation of SPUs significantly improves the overall translation quality by 1.88,1.43, and 3.75 BLEU, respectively. Moreover, detailed analysis shows that our approaches much strengthen the association of source and target words. From the preceding, we can see that our proposed model is effective both in rich- and low-resource scenarios.
机译：在神经机翻译（NMT）中，源和目标词位于大深度神经网络的两端，通常由一系列非线性激活介导。这种非线性激活的问题是它们显着降低了深度神经网络中梯度的大小，从而逐渐松开了源单词与其翻译之间的相互作用。结果，源词可能被错误地翻译成其翻译等同物。在本文中，我们通过允许通过线性插值有效地通过线性插值来加强源单位和目标单词的关联来加强源单位和目标词的关联。特别是，我们丰富了具有spus的三个关键的NMT组件：（1）具有SPU的丰富编码模型，其线性地插入源单词嵌入式源注释; （2）具有SPU的丰富的解码模型，它使源上下文能够线性地流到目标侧隐藏状态; （3）具有SPU的富集的输出模型，进一步允许将目标侧隐藏状态的线性插值插入输出状态。汉语 - 英语，英语到德语和低资源藏语到中文翻译任务的实验表明，Spus的线性插值显着提高了1.88,1.43和3.75 Bleu的整体翻译质量。此外，详细分析表明，我们的方法加强了来源和目标词的关联。从前一开始，我们可以看到我们的提议模型在富裕和低资源方案中都有效。

著录项

相似文献

  • 外文文献
  • 中文文献
  • 专利
获取原文

客服邮箱：kefu@zhangqiaokeyan.com

京公网安备：11010802029741号 ICP备案号：京ICP备15016152号-6 六维联合信息科技 (北京) 有限公司©版权所有

  • 客服微信

  • 服务号