...
  • 主题
高级搜索  >
历史搜索 清空历史
首页> 外文期刊>Signal processing >Performance limits of stochastic sub-gradient learning, Part I: Single agent case
【24h】

Performance limits of stochastic sub-gradient learning, Part I: Single agent case

机译：随机次梯度学习的性能限制，第I部分：单代理案例

获取原文
获取原文并翻译 | 示例
   

获取外文期刊封面封底 >>

       
页面导航
  • 摘要
  • 著录项
  • 相似文献
  • 相关主题

摘要

Highlights•A novel sub-gradient assumption is proposed.•New convergence and steady-state performance is been proved under the diffusion strategy.•Multiple examples with this new assumption combined with the diffusion strategy have been investigated.AbstractThe analysis in Part I [1] revealed interesting properties for subgradient learning algorithms in the context ofstochasticoptimization. These algorithms are used when the risk functions are non-smooth or involve non-differentiable components. They have been long recognized as being slow converging methods. However, it was revealed in Part I that the rate of convergence becomes linear forstochasticoptimization problems, with the error iterate converging at an exponential rateαito within anO(μ)−neighborhood of the optimizer, for someα ∈ (0, 1) and small step-sizeμ. The conclusion was established under weaker assumptions than the prior literature and, moreover, several important problems were shown to satisfy these weaker assumptions automatically. These results revealed that sub-gradient learning methods have more favorable behavior than originally thought. The results of Part I were exclusive to single-agent adaptation. The purpose of current Part II is to examine the implications of these discoveries when a collection of networked agents employs subgradient learning as their cooperative mechanism. The analysis will show that, despite the coupled dynamics that arises in a networked scenario, the agents are still able to attain linear convergence in the stochastic case; they are also able to reach agreement withinO(μ) of the optimizer.
机译： 突出显示 • 提出了一种新颖的次梯度假设。 • 在扩散策略下证明了新的收敛性和稳态性能。 < / ce：list-item> • 与此相关的多个示例研究了结合扩散策略的新假设。 摘要 第一部分[1]中的分析揭示了在随机优化的背景下，次梯度学习算法的有趣特性。当风险函数不平滑或涉及不可微的成分时，将使用这些算法。长期以来，它们一直被认为是缓慢收敛的方法。但是，在第一部分中揭示，对于随机优化问题，收敛速度变为线性，误差迭代以指数速率收敛α i 到 O （ μ ） − 优化器的邻居，对于某些< ce：italic>α∈（0，1）和小步长μ。该结论是在比现有文献更弱的假设下建立的，此外，还显示出一些重要的问题可以自动满足这些较弱的假设。这些结果表明，次梯度学习方法比最初的想法具有更好的行为。第一部分的结果仅适用于单主体适应。当前第二部分的目的是研究当网络代理的集合采用次梯度学习作为其合作机制时这些发现的含义。分析将表明，尽管在网络情况下会出现耦合动力学，但在随机情况下，代理仍然能够实现线性收敛；他们还能够在优化程序的 O （μ）内达成共识。

著录项

相似文献

  • 外文文献
  • 中文文献
  • 专利
获取原文

客服邮箱：kefu@zhangqiaokeyan.com

京公网安备：11010802029741号 ICP备案号：京ICP备15016152号-6 六维联合信息科技 (北京) 有限公司©版权所有

  • 客服微信

  • 服务号