• 主题
高级搜索  >
历史搜索 清空历史
首页> 外文期刊>Neural computation >A Novel Predictive-Coding-Inspired Variational RNN Model for Online Prediction and Recognition
【24h】

A Novel Predictive-Coding-Inspired Variational RNN Model for Online Prediction and Recognition

机译：在线预测和识别的新型预测编码启发式变分RNN模型

获取原文
获取原文并翻译 | 示例
           
页面导航
  • 摘要
  • 著录项
  • 相似文献
  • 相关主题

摘要

This study introduces PV-RNN, a novel variational RNN inspired by predictive-coding ideas. The model learns to extract the probabilistic structures hidden in fluctuating temporal patterns by dynamically changing the stochasticity of its latent states. Its architecture attempts to address two major concerns of variational Bayes RNNs: how latent variables can learn meaningful representations and how the inference model can transfer future observations to the latent variables. PV-RNN does both by introducing adaptive vectors mirroring the training data, whose values can then be adapted differently during evaluation. Moreover, prediction errors during backpropagation-rather than external inputs during the forward computation-are used to convey information to the network about the external data. For testing, we introduce error regression for predicting unseen sequences as inspired by predictive coding that leverages those mechanisms. As in other variational Bayes RNNs, our model learns by maximizing a lower bound on the marginal likelihood of the sequential data, which is composed of two terms: the negative of the expectation of prediction errors and the negative of the Kullback-Leibler divergence between the prior and the approximate posterior distributions. The model introduces a weighting parameter, the meta-prior, to balance the optimization pressure placed on those two terms. We test the model on two data sets with probabilistic structures and show that with high values of the meta-prior, the network develops deterministic chaos through which the randomness of the data is imitated. For low values, the model behaves as a random process. The network performs best on intermediate values and is able to capture the latent probabilistic structure with good generalization. Analyzing the meta-prior's impact on the network allows us to precisely study the theoretical value and practical benefits of incorporating stochastic dynamics in our model. We demonstrate better prediction performance on a robot imitation task with our model using error regression compared to a standard variational Bayes model lacking such a procedure.
机译：这项研究介绍了PV-RNN，这是一种受预测编码思想启发的新型变体RNN。该模型学习通过动态改变其潜在状态的随机性来提取隐藏在波动时间模式中的概率结构。它的体系结构试图解决变数贝叶斯RNN的两个主要问题：潜变量如何学习有意义的表示，以及推理模型如何将未来的观测值转移到潜变量。 PV-RNN都通过引入反映训练数据的自适应矢量来实现，然后可以在评估过程中以不同的方式调整其值。此外，反向传播期间的预测错误（而不是前向计算期间的外部输入）用于将有关外部数据的信息传递给网络。对于测试，我们引入了误差回归来预测看不见的序列，这是受到利用那些机制的预测编码的启发。与其他变分贝叶斯RNN一样，我们的模型通过最大化顺序数据边际可能性的下限来学习，该下限由两个术语组成：预测误差的期望值为负，而预测误差之间的Kullback-Leibler差异为负。先验分布和近似后验分布。该模型引入了一个加权参数meta-prior，以平衡施加在这两项上的优化压力。我们在具有概率结构的两个数据集上测试了该模型，结果表明，在元先验值较高的情况下，网络会发展出确定性混乱，从而可以模仿数据的随机性。对于低值，模型表现为随机过程。网络在中间值上表现最佳，并且能够以良好的概括性捕获潜在的概率结构。通过分析元先验对网络的影响，我们可以精确地研究将随机动力学纳入模型的理论价值和实际收益。与没有这种程序的标准变分贝叶斯模型相比，我们使用误差回归模型对机器人模仿任务展示了更好的预测性能。

著录项

  • 来源
    《Neural computation》 |2019年第11期|2025-2074|共50页
  • 作者

    Ahmadi Ahmadreza; Jun Tani;

  • 作者单位

    Okinawa Inst Sci & Technol Onna Okinawa 9040495 Japan|Korea Adv Inst Sci & Technol Sch Elect Engn Daejeon 305701 South Korea;

    Okinawa Inst Sci & Technol Onna Okinawa 9040495 Japan;

  • 收录信息 美国《科学引文索引》(SCI);美国《化学文摘》(CA);
  • 原文格式 PDF
  • 正文语种 eng
  • 中图分类
  • 关键词

相似文献

  • 外文文献
  • 中文文献
  • 专利
获取原文

客服邮箱：kefu@zhangqiaokeyan.com

京公网安备：11010802029741号 ICP备案号：京ICP备15016152号-6 六维联合信息科技 (北京) 有限公司©版权所有

  • 客服微信

  • 服务号