• 主题
高级搜索  >
历史搜索 清空历史
首页> 外文会议>3rd workshop on representation learning for NLP 2018 >LSTMs Exploit Linguistic Attributes of Data
【24h】

LSTMs Exploit Linguistic Attributes of Data

机译：LSTM利用数据的语言属性

获取原文
获取原文并翻译 | 示例
页面导航
  • 摘要
  • 著录项
  • 相似文献
  • 相关主题

摘要

While recurrent neural networks have found success in a variety of natural language processing applications, they are general models of sequential data. We investigate how the properties of natural language data affect an LSTM's ability to learn a nonlinguistic task: recalling elements from its input. We find that models trained on natural language data are able to recall tokens from much longer sequences than models trained on non-language sequential data. Furthermore, we show that the LSTM learns to solve the memorization task by explicitly using a subset of its neurons to count timesteps in the input. We hypothesize that the patterns and structure in natural language data enable LSTMs to learn by providing approximate ways of reducing loss, but understanding the effect of different training data on the learnability of LSTMs remains an open question.
机译：虽然递归神经网络已在各种自然语言处理应用中获得成功，但它们是顺序数据的通用模型。我们研究自然语言数据的属性如何影响LSTM学习非语言任务的能力：从其输入中回忆元素。我们发现，与在非语言顺序数据上训练的模型相比，在自然语言数据上训练的模型能够从更长的序列中调用令牌。此外，我们显示LSTM通过显式使用其神经元的子集来计算输入中的时间步长，从而学会解决记忆任务。我们假设自然语言数据中的模式和结构通过提供减少损失的近似方法使LSTM能够学习，但是了解不同训练数据对LSTM可学习性的影响仍然是一个悬而未决的问题。

著录项

  • 来源
  • 会议地点 Melbourne(AU)
  • 作者

    Nelson F. Liu; Omer Levy; Roy Schwartz; Chenhao Tan; Noah A. Smith;

  • 作者单位

    Paul G. Allen School of Computer Science Engineering, University of Washington, Seattle, WA, USA,Department of Linguistics, University of Washington, Seattle, WA, USA;

    Paul G. Allen School of Computer Science Engineering, University of Washington, Seattle, WA, USA;

    Paul G. Allen School of Computer Science Engineering, University of Washington, Seattle, WA, USA,Allen Institute for Artificial Intelligence, Seattle, WA, USA;

    Department of Computer Science, University of Colorado, Boulder, CO, USA;

    Paul G. Allen School of Computer Science Engineering, University of Washington, Seattle, WA, USA;

  • 会议组织
  • 原文格式 PDF
  • 正文语种 eng
  • 中图分类
  • 关键词

相似文献

  • 外文文献
  • 中文文献
  • 专利
获取原文

客服邮箱：kefu@zhangqiaokeyan.com

京公网安备：11010802029741号 ICP备案号：京ICP备15016152号-6 六维联合信息科技 (北京) 有限公司©版权所有

  • 客服微信

  • 服务号