Developing Learning Algorithms via Optimized Discretization of Continuous Dynamical Systems

Tao Q.; Sun Z.; Kong K.

首页> 外文期刊>Systems, Man, and Cybernetics, Part B: Cybernetics, IEEE Transactions on >Developing Learning Algorithms via Optimized Discretization of Continuous Dynamical Systems

【24h】

Developing Learning Algorithms via Optimized Discretization of Continuous Dynamical Systems

机译：通过连续动力系统的优化离散化开发学习算法

获取原文

获取原文并翻译 | 示例

掌桥外文数据库（机构版） >>

开具论文收录证明 >>

文献代查 >>

页面导航

摘要
著录项
相似文献
相关主题

摘要

Most of the existing numerical optimization methods are based upon a discretization of some ordinary differential equations. In order to solve some convex and smooth optimization problems coming from machine learning, in this paper, we develop efficient batch and online algorithms based on a new principle, i.e., the optimized discretization of continuous dynamical systems (ODCDSs). First, a batch learning projected gradient dynamical system with Lyapunov's stability and monotonic property is introduced, and its dynamical behavior guarantees the accuracy of discretization-based optimizer and applicability of line search strategy. Furthermore, under fair assumptions, a new online learning algorithm achieving regret $O(sqrt{T})$ or $O(logT)$ is obtained. By using the line search strategy, the proposed batch learning ODCDS exhibits insensitivity to the step sizes and faster decrease. With only a small number of line search steps, the proposed stochastic algorithm shows sufficient stability and approximate optimality. Experimental results demonstrate the correctness of our theoretical analysis and efficiency of our algorithms.

机译：大多数现有的数值优化方法都是基于一些常微分方程的离散化。为了解决机器学习带来的一些凸且平滑的优化问题，在本文中，我们基于新原理（即连续动态系统（ODCDS）的优化离散化）开发了高效的批处理和在线算法。首先，介绍了具有Lyapunov稳定性和单调性的批量学习投影梯度动力学系统，其动力学行为保证了基于离散化的优化器的准确性和线性搜索策略的适用性。此外，在公平的假设下，获得了一种实现后悔$ O（sqrt {T}）$或$ O（logT）$的新在线学习算法。通过使用线搜索策略，建议的批处理学习ODCDS表现出对步长不敏感并且更快地减小。仅用少量的线搜索步骤，所提出的随机算法就显示出足够的稳定性和近似最优性。实验结果证明了我们理论分析的正确性和算法的有效性。

著录项

来源
《Systems, Man, and Cybernetics, Part B: Cybernetics, IEEE Transactions on》 |2012年第1期|p.140-149|共10页
作者
Tao Q.; Sun Z.; Kong K.;
展开▼
作者单位

Institute of Automation, Chinese Academy of Sciences, Beijing, China;

展开▼
收录信息
原文格式 PDF
正文语种 eng
中图分类
关键词
Dynamical systems; Regret; line search; machine learning; online learning; optimization algorithms; projected subgradient algorithms;

机译：动力系统;遗憾;在线搜索;机器学习;在线学习;优化算法;投影次梯度算法;

相似文献

外文文献
中文文献
专利

1. Evaluation of genetic algorithms using discrete and continuous methods for pump optimization of water distribution systems [J] . Seyed Mohsen Sadatiyan Abkenar, Samuel Dustin Stanley, Carol J. Miller, Sustainable Computing . 2015,第DECa期

机译：使用离散和连续方法评估供水系统泵优化的遗传算法
2. Continuous and Discrete Composite Adjoints for the Hessian of the Lagrangian in Shooting Algorithms for Dynamic Optimization [J] . Hannemann Ralf, Marquardt Wolfgang SIAM Journal on Scientific Computing . 2010,第6期

机译：动态优化射击算法中拉格朗日黑森州连续离散离散复合伴随
3. Continuous dynamical systems that realize discrete optimization on the hypercube [J] . Absil PA, Sepulchre R Systems and Control Letters . 2004,第3a4期

机译：在超立方体上实现离散优化的连续动力学系统
4. Some Implications of System Dynamics Analysis of Discrete-Time Recurrent Neural Networks for Learning Algorithms Design [C] . Jorge Cervantes, Maria Gomez, Alexander Schaum Mexican International Conference on Artificial Intelligence . 2013

机译：自由时间经常性神经网络学习算法设计的系统动力学分析的一些影响
5. Algorithms for mixed continuous-discrete variable problems in structural optimization. [D] . Huang, Min-Wei. 1995

机译：结构优化中混合连续离散变量问题的算法。
6. Modified differential evolution algorithm with onlooker bee operator for mixed discrete-continuous optimization [O] . Yongfei Miao, Qinghua Su, Zhongbo Hu, -1

机译：带有旁观蜂算子的改进差分进化算法用于混合离散连续优化
7. DIGITAL CONTROL SYSTEMS IMPLEMENTATION TECHNIQUES, VOLUME 70 OF CONTROL AND DYNAMIC SYSTEMS: ADVANCES IN THEORY AND APPLICATIONS, C. Leondes (ed), Academic Press, San Diego, 1995, 390 pp., ISBN 0-12-0127702, $99.00 DISCRETE-TIME CONTROL SYSTEM ANALYSIS AND DESIGN, VOLUME 71 OF CONTROL AND DYNAMIC SYSTEMS: ADVANCES IN THEORY AND APPLICATIONS, C. Leondes (ed), Academic Press, San Diego, 1995, 410 pp., ISBN 0-12-0127715, $99.00 DISCRETE-TIME CONTROL SYSTEM IMPLEMENTATION TECHNIQUES, VOLUME 72 OF CONTROL AND DYNAMIC SYSTEMS: ADVANCES IN THEORY AND APPLICATIONS, C. Leondes (ed), Academic Press, San Diego, 1995, 388 pp., ISBN 0-12-0127725, $99.00 TECHNIQUES IN DISCRETE-TIME STOCHASTIC CONTROL SYSTEMS, VOLUME 73 OF CONTROL AND DYNAMIC SYSTEMS: ADVANCES IN THEORY AND APPLICATIONS, C. Leondes (ed), Academic Press, San Diego, 1995, 380 pp., ISBN 0-12-0127734, $99.00 TECHNIQUES IN DISCRETE AND CONTINUOUS ROBUST SYSTEMS, VOLUME 74 OF CONTROL AND DYNAMIC SYSTEMS: ADVANCES IN THEORY AND APPLICATIONS, C. Leondes (ed), Academic Press, San Diego, 1995, 412 pp., ISBN 0-12-0127741, $99.00 [O] . D. SUBBARAM NAIDU 1997

机译：数字控制系统的实施技术，控制和动态系统的第70卷：理论和应用的进步，C.丝丝（ED），学术出版社，圣地亚哥，1995,390 PP，ISBN 0-12-0127702，99.00美元离散时间控制系统分析和设计，控制和动态系统的第71卷：理论和应用的进步，C.丝丝（ED），学术出版社，圣地亚哥，1995,410 PP，ISBN 0-12-0127715，99.00美元离散时间控制系统实现技术，控制和动态系统的第72卷：理论和应用的进步，C.丝丝（ED），学术出版社，圣地亚哥，1995,388 pp，ISBN 0-12-0127725，99.00美元的离散 - 时间随机控制系统，控制和动态系统的VOLUME 73：前进，理论与应用，C Leondes（ED），学术出版社，圣地亚哥，1995年，380页，ISBN 0-12-0127734，在离散和$ 99.00技术。连续稳健的系统，控制和动态系统的第74卷：理论和应用的进步，C. leondes（ed），学术出版社，圣地亚哥，1995,412 pp。，ISBN 0-12-0127741，99.00美元

Developing Learning Algorithms via Optimized Discretization of Continuous Dynamical Systems

摘要

著录项

相似文献

相关主题

期刊订阅