• 主题
高级搜索  >
历史搜索 清空历史
首页> 外文会议>International Conference on Machine Learning >On the Convergence of Nesterov's Accelerated Gradient Method in Stochastic Settings
【24h】

On the Convergence of Nesterov's Accelerated Gradient Method in Stochastic Settings

机译：在随机设置中Nesterov加速梯度法的融合

获取原文
页面导航
  • 摘要
  • 著录项
  • 相似文献
  • 相关主题

摘要

We study Nesterov's accelerated gradient method with constant step-size and momentum parameters in the stochastic approximation setting (unbiased gradients with bounded variance) and the finite-sum setting (where randomness is due to sampling mini-batches). To build better insight into the behavior of Nesterov's method in stochastic settings, we focus throughout on objectives that are smooth, strongly-convex, and twice continuously differentiable. In the stochastic approximation setting, Nesterov's method converges to a neighborhood of the optimal point at the same accelerated rate as in the deterministic setting. Perhaps surprisingly, in the finite-sum setting, we prove that Nesterov's method may diverge with the usual choice of step-size and momentum, unless additional conditions on the problem related to conditioning and data coherence are satisfied. Our results shed light as to why Nesterov's method may fail to converge or achieve acceleration in the finite-sum setting.
机译：我们在随机近似设置中的恒定步长和动量参数（具有有界差异的非偏见梯度）和有限和设置（其中由于采样小批次而导致的偏差梯度）来研究Nesterov的加速梯度方法。 为了更好地了解Nesterov在随机设置中的方法的行为，我们整体上专注于流畅，强烈凸出的目标，两次连续可差。 在随机近似设置中，Nesterov的方法会聚到与确定性设置相同加速速率的最佳点的邻域。 也许令人惊讶的是，在有限和设置中，我们证明了Nesterov的方法可能随着常规选择的阶梯大小和势头而分歧，除非满足有关与调理和数据一致性相关的问题的额外条件。 我们的结果阐明了为什么Nesterov的方法可能无法在有限和设置中收敛或实现加速度。

著录项

相似文献

  • 外文文献
  • 中文文献
  • 专利
获取原文

客服邮箱：kefu@zhangqiaokeyan.com

京公网安备：11010802029741号 ICP备案号：京ICP备15016152号-6 六维联合信息科技 (北京) 有限公司©版权所有

  • 客服微信

  • 服务号