Segmented-Memory Recurrent Neural Networks

Jinmiao Chen; Chaudhari N.S.

首页> 外文期刊>Neural Networks, IEEE Transactions on >Segmented-Memory Recurrent Neural Networks

【24h】

Segmented-Memory Recurrent Neural Networks

机译：分段记忆递归神经网络

获取原文

获取原文并翻译 | 示例

获取外文期刊封面封底 >>

开具论文收录证明 >>

文献代查 >>

页面导航

摘要
著录项
相似文献
相关主题

摘要

Conventional recurrent neural networks (RNNs) have difficulties in learning long-term dependencies. To tackle this problem, we propose an architecture called segmented-memory recurrent neural network (SMRNN). A symbolic sequence is broken into segments and then presented as inputs to the SMRNN one symbol per cycle. The SMRNN uses separate internal states to store symbol-level context, as well as segment-level context. The symbol-level context is updated for each symbol presented for input. The segment-level context is updated after each segment. The SMRNN is trained using an extended real-time recurrent learning algorithm. We test the performance of SMRNN on the information latching problem, the ldquotwo-sequence problemrdquo and the problem of protein secondary structure (PSS) prediction. Our implementation results indicate that SMRNN performs better on long-term dependency problems than conventional RNNs. Besides, we also theoretically analyze how the segmented memory of SMRNN helps learning long-term temporal dependencies and study the impact of the segment length.

机译：传统的递归神经网络（RNN）在学习长期依赖性方面有困难。为了解决这个问题，我们提出了一种称为分段内存递归神经网络（SMRNN）的体系结构。一个符号序列被分成多个段，然后作为输入提供给SMRNN，每个周期一个符号。 SMRNN使用单独的内部状态来存储符号级上下文和段级上下文。对于输入的每个符号，将更新符号级别的上下文。在每个段之后更新段级别上下文。 SMRNN使用扩展的实时递归学习算法进行训练。我们测试了SMRNN在信息锁存问题，“二序列问题”和蛋白质二级结构（PSS）预测问题上的性能。我们的实施结果表明，SMRNN在长期依赖问题上比常规RNN表现更好。此外，我们还从理论上分析了SMRNN的分段记忆如何帮助学习长期的时间依赖性并研究分段长度的影响。

著录项

来源
《Neural Networks, IEEE Transactions on》 |2009年第8期|p.1267-1280|共14页
作者
Jinmiao Chen; Chaudhari N.S.;
展开▼
作者单位

Sch. of Comput. Eng., Nanyang Technol. Univ., Singapore, Singapore;

展开▼
收录信息
原文格式 PDF
正文语种 eng
中图分类
关键词
learning (artificial intelligence); recurrent neural nets; RNN; extended real-time recurrent learning algorithm; information latching problem; long-term temporal dependency learning; protein secondary structure prediction; segment-level context; segmented-memory recurrent neural network training; symbol-level context; symbolic sequence; two-sequence problem; Gradient descent; information latching; long-term dependencies; recurrent neural networks (RNNs); segmented memory; vanishing gradient;

机译：学习（人工智能）;递归神经网络;RNN;扩展的实时递归学习算法;信息锁存问题;长期时间依赖学习;蛋白质二级结构预测;段级上下文;分段内存递归神经网络训练;符号级上下文;符号序列;二序列问题;梯度下降;信息锁存;长期依赖;递归神经网络（RNN）;分段记忆;消失梯度;

相似文献

外文文献
中文文献
专利

1. Learning long-term dependencies in segmented-memory recurrent neural networks with backpropagation of error [J] . Stefan Gluege, Ronald Boeck, Guenther Palm, Neurocomputing . 2014,第octa2期

机译：在错误的反向传播下学习分段记忆递归神经网络中的长期依赖性
2. Bidirectional segmented-memory recurrent neural network for protein secondary structure prediction [J] . Chen J, Chaudhari NS Soft computing: A fusion of foundations, methodologies and applications . 2006,第4期

机译：双向分段记忆递归神经网络用于蛋白质二级结构预测
3. Rethinking the Combined and Individual Orders of Derivative of States for Differential Recurrent Neural Networks: Deep Differential Recurrent Neural Networks [J] . NAIFAN ZHUANG, GUO-JUN QI, THE DUC KIEU, ACM transactions on multimedia computing communications and applications . 2019,第3期

机译：重新思考各国衍生物的衍生物的衍生物的衍生物：深度差分经常性神经网络
4. Auto-Encoder Pre-Training of Segmented-Memory Recurrent Neural Networks [C] . Stefan Gluege, Ronald Boeck, Andreas Wendemuth European symposium on artificial neural networks, computational intelligence and machine learning . 2013

机译：分段记忆递归神经网络的自动编码器预训练
5. Gene expression temporal patterns classification with hierarchical Bayesian neural networks and time lagged recurrent neural networks. [D] . Liang, Yulan. 2003

机译：利用分层贝叶斯神经网络和时滞递归神经网络对基因表达时间模式进行分类。
6. Finding the optical properties of plasmonic structures by image processing using a combination of convolutional neural networks and recurrent neural networks [O] . Iman Sajedian, Jeonghyun Kim, Junsuk Rho 2019

机译：结合卷积神经网络和递归神经网络进行图像处理找到等离激元结构的光学性质
7. Recognizing recurrent neural networks (rRNN): Bayesian inference for recurrent neural networks [O] . Sebastian Bitzer, Stefan J. Kiebel 2012

机译：识别递归神经网络（rRNN）：递归神经网络的贝叶斯推断

Segmented-Memory Recurrent Neural Networks

摘要

著录项

相似文献

相关主题

期刊订阅