• 主题
高级搜索  >
历史搜索 清空历史
首页> 外文会议>Conference on the North American Chapter of the Association for Computational Linguistics: Human Language Technologies >Studying the Inductive Biases of RNNs with Synthetic Variations of Natural Languages
【24h】

Studying the Inductive Biases of RNNs with Synthetic Variations of Natural Languages

机译：用自然语言的合成变化研究RNN的归纳偏差

获取原文
页面导航
  • 摘要
  • 著录项
  • 相似文献
  • 相关主题

摘要

How do typological properties such as word order and morphological case marking affect the ability of neural sequence models to acquire the syntax of a language? Cross-linguistic comparisons of RNNs' syntactic performance (e.g., on subject-verb agreement prediction) are complicated by the fact that any two languages differ in multiple typological properties, as well as by differences in training corpus. We propose a paradigm that addresses these issues: we create synthetic versions of English, which differ from English in one or more typological parameters, and generate corpora for those languages based on a parsed English corpus. We report a series of experiments in which RNNs were trained to predict agreement features for verbs in each of those synthetic languages. Among other findings, (1) performance was higher in subject-verb-object order (as in English) than in subject-object-verb order (as in Japanese), suggesting that RNNs have a recency bias; (2) predicting agreement with both subject and object (polypersonal agreement) improves over predicting each separately, suggesting that underlying syntactic knowledge transfers across the two tasks: and (3) overt morphological case makes agreement prediction significantly easier, regardless of word order.
机译：如何如何进行类型的类型，例如Word Order和Morphological Case标记影响神经序列模型获取语言语法的能力？ RNNS句法性能的交叉语言比较（例如，关于主题 - 动词协议预测）的事实是多种类型学特性的任何两种语言的事实，以及培训语料库的差异。我们提出了一个解决这些问题的范例：我们创建了英语的合成版本，它在一个或多个类型的参数中不同于英语，并根据有解析的英语语料库生成这些语言的Corpora。我们报告了一系列实验，其中RNN培训，以预测每个合成语言中的每个综合语的动词的协议功能。在其他发现之外，（1）性能在主题 - 动词 - 对象顺序（如英文）中比在主题 - 对象 - 动词秩序（如日语中），表明RNN具有新近偏见; （2）与主题和对象（多蛋白交易协议）预测协议改善了每分别预测，这表明涉及两项任务的奇妙知识转移：（3）明显的形态案件使得协议预测明显更容易，无论单词订单如何更容易。

著录项

相似文献

  • 外文文献
  • 中文文献
  • 专利
获取原文

客服邮箱：kefu@zhangqiaokeyan.com

京公网安备：11010802029741号 ICP备案号：京ICP备15016152号-6 六维联合信息科技 (北京) 有限公司©版权所有

  • 客服微信

  • 服务号