A distributional code for value in dopamine- based reinforcement learning

Dabney Will; Kurth-Nelson Zeb; Uchida Naoshige; Starkweather Clara Kwon; Hassabis Demis; Munos Remi; Botvinick Matthew

首页> 外文期刊>Nature >A distributional code for value in dopamine- based reinforcement learning

【24h】

A distributional code for value in dopamine- based reinforcement learning

机译：基于多巴胺的强化学习中的价值分配代码

获取原文

获取原文并翻译 | 示例

掌桥外文数据库（机构版） >>

开具论文收录证明 >>

文献代查 >>

页面导航

摘要
著录项
相似文献
相关主题

摘要

Since its introduction, the reward prediction error theory of dopamine has explained a wealth of empirical phenomena, providing a unifying framework for understanding the representation of reward and value in the brain(1-3). According to the now canonical theory, reward predictions are represented as a single scalar quantity, which supports learning about the expectation, or mean, of stochastic outcomes. Here we propose an account of dopamine-based reinforcement learning inspired by recent artificial intelligence research on distributional reinforcement learning(4-6). We hypothesized that the brain represents possible future rewards not as a single mean, but instead as a probability distribution, effectively representing multiple future outcomes simultaneously and in parallel. This idea implies a set of empirical predictions, which we tested using single-unit recordings from mouse ventral tegmental area. Our findings provide strong evidence for a neural realization of distributional reinforcement learning.

机译：自引入以来，多巴胺的奖励预测误差理论已经解释了许多经验现象，为理解大脑中奖励和价值的表示提供了统一的框架（1-3）。根据现在的规范理论，报酬预测表示为单个标量，它支持了解随机结果的期望值或均值。在这里，我们提出了一种基于多巴胺的强化学习的方法，该方法受最近关于分布式强化学习的人工智能研究的启发（4-6）。我们假设大脑不是以均值的方式代表未来可能的回报，而是以概率分布的方式来代表，可以有效地同时并行地代表多个未来成果。这个想法暗示了一组经验预测，我们使用来自小鼠腹侧被盖区的单个单位记录进行了测试。我们的发现为神经网络实现分布强化学习提供了有力的证据。

著录项

来源
《Nature》 |2020年第7792期|671-675|共5页
作者
Dabney Will; Kurth-Nelson Zeb; Uchida Naoshige; Starkweather Clara Kwon; Hassabis Demis; Munos Remi; Botvinick Matthew;
展开▼
作者单位

DeepMind London England;

DeepMind London England|UCL Max Planck UCL Ctr Computat Psychiat & Ageing Res London England;

Harvard Univ Dept Mol & Cellular Biol Ctr Brain Sci Cambridge MA 02138 USA;

DeepMind London England|UCL Gatsby Computat Neurosci Unit London England;

展开▼
收录信息美国《科学引文索引》(SCI);美国《工程索引》(EI);美国《生物学医学文摘》(MEDLINE);美国《化学文摘》(CA);
原文格式 PDF
正文语种 eng
中图分类
关键词

相似文献

外文文献
中文文献
专利

1. Optimizing MDS-coded cache-enable wireless network:a blockchain-based cooperative deep reinforcement learning approach [J] . Zhang Zheng, Yang Ruizhe, Yu Fei Richard, 高技术通讯（英文版） . 2021,第002期

机译：优化MDS编码的缓存 - 使能无线网络：基于区块链的合作深度增强学习方法
2. Reinforcement learning based coding unit early termination algorithm for high efficiency video coding [J] . Li Na, Zhang Yun, Zhu Linwei, Journal of visual communication & image representation . 2019,第Apra期

机译：基于加强学习的编码单元高效视频编码的早期终止算法
3. Deep Distributional Reinforcement Learning Based High-Level Driving Policy Determination [J] . Kyushik Min, Hayoung Kim, Kunsoo Huh IEEE Transactions on Intelligent Vehicles . 2019,第3期

机译：基于深度分布强化学习的高级驾驶策略确定
4. Optimizing Degree Distributions of LT-based Codes with Deep Reinforcement Learning [C] . Yehor Savchenko, Yi Liu IEEE Conference on Computer Communications Workshops . 2019

机译：深度强化学习优化基于LT的代码的度分布
5. Understanding Model-Based Reinforcement Learning and its Application in Safe Reinforcement Learning [D] . Hu, Dingcheng . 2019

机译：了解基于模型的强化学习及其在安全强化学习中的应用
6. Reinforcement Learning Based Fast Self-Recalibrating Decoder for Intracortical Brain–Machine Interface [O] . Peng Zhang, Lianying Chao, Yuting Chen, 2020

机译：基于加强学习的内部脑机接口快速自我重新校准解码器
7. Reinforcement Learning Based Cooperative Coded Caching Under Dynamic Popularities in Ultra-Dense Networks [O] . Shen Gao, Peihao Dong, Zhiwen Pan, 2020

机译：基于强化学习的超密集网络动态普及下的合作编码缓存

A distributional code for value in dopamine- based reinforcement learning

摘要

著录项

相似文献

相关主题

期刊订阅