• 主题
高级搜索  >
历史搜索 清空历史
首页> 外文期刊>International Journal of Innovative Computing Information and Control >LANGUAGE MODELING USING AUGMENTED ECHO STATE NETWORKS
【24h】

LANGUAGE MODELING USING AUGMENTED ECHO STATE NETWORKS

机译：使用增强的回声状态网络进行语言建模

获取原文
获取原文并翻译 | 示例
       
页面导航
  • 摘要
  • 著录项
  • 相似文献
  • 相关主题

摘要

Interest in natural language modeling using neural networks has been growing in the past decade. The objective of this paper is to investigate the predictive capabilities of echo state networks (ESNs) at the task of modeling English sentences. Based on the finding that ESNs exhibit a Markovian organization of their state space that makes them close to the widely used n-gram models, we describe two modifications of the conventional architecture that allow significant improvement by leveraging the kind of representation developed in the reservoir. Firstly, the addition of pre-recurrent features is shown to capture syntactic similarities between words and can be trained efficiently by using the contracting property of the reservoir to truncate the gradient descent. Secondly, the addition of multiple linear readouts using the mixture of experts framework is also shown to greatly improve accuracy while being trainable in parallel using Expectation-Maximization. Furthermore it can easily be transformed into a supervised mixture of expert model with several variations allowing reducing the training time and can take into account handmade features.
机译：在过去的十年中，人们对使用神经网络进行自然语言建模的兴趣不断增长。本文的目的是研究以英语句子建模为任务的回声状态网络（ESN）的预测能力。基于ESN表现出状态空间的马尔可夫组织（使它们接近于广泛使用的n-gram模型）的发现，我们描述了常规体系结构的两种修改形式，这些形式可以利用储层中开发的表示形式进行显着改进。首先，显示出递归特征的添加可以捕获单词之间的句法相似性，并且可以通过使用水库的收缩特性来截断梯度下降而有效地对其进行训练。其次，使用混合专家框架添加多个线性读数还可以显着提高准确性，同时可以使用Expectation-Maximization进行并行训练。此外，它可以很容易地转换成具有多种变化的专家模型的受监督混合，从而减少了培训时间，并且可以考虑手工功能。

著录项

相似文献

  • 外文文献
  • 中文文献
  • 专利
获取原文

客服邮箱：kefu@zhangqiaokeyan.com

京公网安备：11010802029741号 ICP备案号：京ICP备15016152号-6 六维联合信息科技 (北京) 有限公司©版权所有

  • 客服微信

  • 服务号