An Efficient Model-Free Approach for Controlling Large-Scale Canals via Hierarchical Reinforcement Learning

Ren Tao; Niu Jianwei; Liu Xuefeng; Wu Jiyan; Lei Xiaohui; Zhang Zhao

首页> 外文期刊>IEEE transactions on industrial informatics >An Efficient Model-Free Approach for Controlling Large-Scale Canals via Hierarchical Reinforcement Learning

【24h】

An Efficient Model-Free Approach for Controlling Large-Scale Canals via Hierarchical Reinforcement Learning

机译：一种有效的无模型方法，用于通过分层加固学习控制大型运河

获取原文

获取原文并翻译 | 示例

掌桥外文数据库（机构版） >>

开具论文收录证明 >>

文献代查 >>

页面导航

摘要
著录项
相似文献
相关主题

摘要

Large-scale canals with cascaded pools are constructed wordwide to divert water from rich to arid areas to mitigate water shortages. Efficient control of canals is essential to improve water-diversion performance. Numerous model-based approaches have been proposed and made great progress for canal control. However, when the predictive model is unavailable or unpromising for long time step predictions, model-free approaches could be considered as a possible way to achieve efficient control. Since most existing model-free approaches are focused on control of small canals or reservoirs, this article proposes a new control approach named policy and action reinforcement learning (PARL) for large-scale canals. We leverage the idea of "divide and conquer" to decompose the control task of large-scale canals into policy learning and action learning subtasks, and develop PARL by means of hierarchical reinforcement learning. Extensive experiments are conducted via numerical simulation on the case study of Chinese South to North Water Transfer Project, and experimental results show that PARL can achieve desirable performance improvements over other model-free learning approaches.

机译：带级联泳池的大型运河被编织方式，将水从富人转移到干旱地区，以减轻水资源短缺。有效控制运河对于改善水导流性能至关重要。已经提出了许多基于模型的方法，并对运河控制取得了很大进展。然而，当预测模型不可用或不妥协时长时间步骤预测，可以将模型方法视为实现有效控制的可能方法。由于大多数现有的无模式方法都专注于对小型运河或水库的控制，因此本文提出了一种为大型运河提供了一个名为政策和行动强化学习（Parl）的新的控制方法。我们利用“鸿沟和征服”的想法，将大规模运河的控制任务分解为政策学习和行动学习子组织，并通过分层加强学习开发Parl。通过数值模拟进行广泛的实验，对中国南部到北水转移项目的案例研究，实验结果表明，PARL可以通过其他无模型学习方法实现所需的性能改进。

著录项

来源
《IEEE transactions on industrial informatics》 |2021年第6期|4367-4378|共12页
作者
Ren Tao; Niu Jianwei; Liu Xuefeng; Wu Jiyan; Lei Xiaohui; Zhang Zhao;
展开▼
作者单位

Beihang Univ Sch Comp Sci & Engn State Key Lab Virtual Real Technol & Syst Beijing 100191 Peoples R China|Beihang Univ Sch Comp Sci & Engn Beijing Adv Innovat Ctr Big Data & Brain Comp BDB Beijing 100191 Peoples R China|Beihang Univ Hangzhou Innovat Inst Hangzhou 310051 Peoples R China;

Beihang Univ Sch Comp Sci & Engn State Key Lab Virtual Real Technol & Syst Beijing 100191 Peoples R China|Beihang Univ Sch Comp Sci & Engn Beijing Adv Innovat Ctr Big Data & Brain Comp BDB Beijing 100191 Peoples R China|Beihang Univ Hangzhou Innovat Inst Hangzhou 310051 Peoples R China|Zhengzhou Univ Res Inst Ind Technol Zhengzhou 450001 Peoples R China;

Beihang Univ Sch Comp Sci & Engn State Key Lab Virtual Real Technol & Syst Beijing 100191 Peoples R China|Beihang Univ Sch Comp Sci & Engn Beijing Adv Innovat Ctr Big Data & Brain Comp BDB Beijing 100191 Peoples R China|Beihang Univ Hangzhou Innovat Inst Hangzhou 310051 Peoples R China;

Beihang Univ Sch Comp Sci & Engn State Key Lab Virtual Real Technol & Syst Beijing 100191 Peoples R China|Beihang Univ Sch Comp Sci & Engn Beijing Adv Innovat Ctr Big Data & Brain Comp BDB Beijing 100191 Peoples R China|Beihang Univ Hangzhou Innovat Inst Hangzhou 310051 Peoples R China;

China Inst Water Resources & Hydropower Res Beijing 100038 Peoples R China;

China Inst Water Resources & Hydropower Res Beijing 100038 Peoples R China;

展开▼
收录信息
原文格式 PDF
正文语种 eng
中图分类
关键词
Irrigation; Water resources; Logic gates; Predictive models; Reinforcement learning; Genetic algorithms; Numerical models; Canal control; hierarchical reinforcement learning (HRL); model-free; multipool; south to north water transfer project;

机译：灌溉;水资源;逻辑门;预测模型;加固学习;遗传算法;数值模型;运河控制;分层加强学习（HRL）;无泡;多池;南到北方水转移项目;

相似文献

外文文献
中文文献
专利

1. An application of multi-objective reinforcement learning for efficient model-free control of canals deployed with IoT networks [J] . Ren Tao, Niu Jianwei, Cui Jiahe, Journal of network and computer applications . 2021,第May期

机译：多目标强化学习对IOT网络部署的运河有效模型控制的应用
2. Model-free reinforcement learning approach to optimal speed control of combustion engines in start-up mode [J] . Zhenhui Xu, Linjun Pan, Tielong Shen Control Engineering Practice . 2021,第Juna期

机译：自由模型加固学习方法启动模式中燃烧发动机的最佳速度控制
3. Three-level hierarchical model-free learning approach to trajectory tracking control [J] . Mircea-Bogdan Radac, Radu-Emil Precup Engineering Applications of Artificial Intelligence . 2016,第octa期

机译：三级无模型学习轨迹跟踪控制方法
4. Hierarchical Control Architecture Regulating Competition between Model-Based and Context-Dependent Model-Free Reinforcement Learning Strategies [C] . Dongjae Kim, Geon Young Park, Sang Wan Lee International Conference on Systems, Man, and Cybernetics . 2018

机译：分层控制体系结构，用于调节基于模型和上下文相关的无模型强化学习策略之间的竞争
5. Dynamic tuning of PI-controllers based on model-free Reinforcement Learning methods. [D] . Abbasi Brujeni, Lena. 2010

机译：基于无模型强化学习方法的PI控制器的动态调整。
6. Control of neural systems at multiple scales using model-free deep reinforcement learning [O] . B. A. Mitchell, L. R. Petzold -1

机译：使用无模型的深度强化学习以多尺度控制神经系统
7. Hierarchical Dynamic Power Management Using Model-Free Reinforcement Learning [O] . Yanzhi Wang, Maryam Triki, Xue Lin, 2014

机译：使用无模型强化学习的分层动态电源管理

An Efficient Model-Free Approach for Controlling Large-Scale Canals via Hierarchical Reinforcement Learning

摘要

著录项

相似文献

相关主题

期刊订阅