Dynamic Cortex Memory: Enhancing Recurrent Neural Networks for Gradient-Based Sequence Learning

机译：动态皮质记忆：增强递归神经网络，用于基于梯度的序列学习

获取原文

页面导航

摘要
著录项
相似文献
相关主题

摘要

In this paper a novel recurrent neural network (RNN) model for gradient-based sequence learning is introduced. The presented dynamic cortex memory (DCM) is an extension of the well-known long short term memory (LSTM) model. The main innovation of the DCM is the enhancement of the inner interplay of the gates and the error carousel due to several new and trainable connections. These connections enable a direct signal transfer from the gates to one another. With this novel enhancement the networks are able to converge faster during training with back-propagation through time (BPTT) than LSTM under the same training conditions. Furthermore, DCMs yield better generalization results than LSTMs. This behaviour is shown for different supervised problem scenarios, including storing precise values, adding and learning a context-sensitive grammar.

机译：本文介绍了一种用于基于梯度的序列学习的新型递归神经网络（RNN）模型。提出的动态皮质记忆（DCM）是众所周知的长期短期记忆（LSTM）模型的扩展。 DCM的主要创新是由于几个新的可训练的连接而增强了闸门的内部相互作用和错误传送带。这些连接使从门到另一门的直接信号传输成为可能。通过这种新颖的增强功能，在相同的训练条件下，与LSTM相比，网络在训练过程中通过时间反向传播（BPTT）能够更快地收敛。此外，DCM比LSTM产生更好的泛化结果。针对不同的有监督问题场景显示了此行为，包括存储精确值，添加和学习上下文相关语法。

著录项

来源
《International conference on artificial neural networks》|2014年|1-8|共8页
会议地点
作者
Sebastian Otte; Marcus Liwicki; Andreas Zell;
展开▼
作者单位

展开▼
会议组织
原文格式 PDF
正文语种
中图分类
关键词
Dynamic Cortex Memory (DCM); Recurrent Neural Networks (RNN); Neural Networks; Long Short Term Memory (LSTM);

机译：动态皮质内存（DCM）;递归神经网络（RNN）;神经网络;长短期记忆（LSTM）;

相似文献

外文文献
中文文献
专利

1. A Novel Fractional Gradient-Based Learning Algorithm for Recurrent Neural Networks [J] . Khan Shujaat, Ahmad Jawwad, Naseem Imran, Circuits, systems, and signal processing . 2018,第2期

机译：一种新的基于分数梯度的递归神经网络学习算法
2. Inhomogeneous Sparseness Leads to Dynamic Instability During Sequence Memory Recall in a Recurrent Neural Network Model [J] . Daniel Medina, Christian Leibold The Journal of Mathematical Neuroscience . 2013,第1期

机译：递归神经网络模型中不均匀的稀疏性导致序列记忆调用期间的动态不稳定
3. Inhomogeneous Sparseness Leads to Dynamic Instability During Sequence Memory Recall in a Recurrent Neural Network Model [J] . Daniel Medina, Christian Leibold The Journal of Mathematical Neuroscience . 2013,第1期

机译：递归神经网络模型中不均匀的稀疏性导致序列记忆调用期间的动态不稳定
4. Dynamic Cortex Memory: Enhancing Recurrent Neural Networks for Gradient-Based Sequence Learning [C] . Sebastian Otte, Marcus Liwicki, Andreas Zell International Conference on Artificial Neural Networks . 2014

机译：动态皮质内存：增强基于梯度的序列学习的经常性神经网络
5. A learning automaton approach to trajectory learning and control system design using dynamic recurrent neural networks. [D] . Condarcure, Thomas A. 1993

机译：一种使用动态递归神经网络进行轨迹学习和控制系统设计的学习自动机方法。
6. Biologically plausible learning in recurrent neural networks reproduces neural dynamics observed during cognitive tasks [O] . Thomas Miconi 2017

机译：循环神经网络中的生物学上合理的学习重现了认知任务期间观察到的神经动力学
7. Inhomogeneous Sparseness Leads to Dynamic Instability During Sequence Memory Recall in a Recurrent Neural Network Model [O] . 2013

机译：递归神经网络模型中不均匀的稀疏性导致序列记忆调用期间的动态不稳定

Dynamic Cortex Memory: Enhancing Recurrent Neural Networks for Gradient-Based Sequence Learning

摘要

著录项

相似文献

相关主题

期刊订阅