...
  • 主题
高级搜索  >
历史搜索 清空历史
首页> 外文期刊>JMLR: Workshop and Conference Proceedings >Stochastic Difference of Convex Algorithm and its Application to Training Deep Boltzmann Machines
【24h】

Stochastic Difference of Convex Algorithm and its Application to Training Deep Boltzmann Machines

机译：凸算法的随机差异及其在训练深玻尔兹曼机中的应用

获取原文
           
页面导航
  • 摘要
  • 著录项
  • 相似文献
  • 相关主题

摘要

Difference of convex functions (DC) programming is an important approach to nonconvex optimization problems because these structures can be encountered in several fields. Effective optimization methods, called DC algorithms, have been developed in deterministic optimization literature. In machine learning, a lot of important learning problems such as the Boltzmann machines (BMs) can be formulated as DC programming. However, there is no DC-like algorithm guaranteed by convergence rate analysis for stochastic problems that are more suitable settings for machine learning tasks. In this paper, we propose a stochastic variant of DC algorithm and give computational complexities to converge to a stationary point under several situations. Moreover, we show our method includes expectation-maximization (EM) and Monte Carlo EM (MCEM) algorithm as special cases on training BMs. In other words, we extend EM/MCEM algorithm to more effective methods from DC viewpoint with theoretical convergence guarantees. Experimental results indicate that our method performs well for training binary restricted Boltzmann machines and deep Boltzmann machines without pre-training.
机译：凸函数（DC）编程的差异是解决非凸优化问题的一种重要方法，因为这些结构可以在多个领域中遇到。在确定性优化文献中已经开发了称为DC算法的有效优化方法。在机器学习中，许多重要的学习问题（例如玻耳兹曼机器（BM））可以表述为DC编程。但是，对于收敛速度分析，并没有针对随机问题的类DC算法，该算法更适合用于机器学习任务。在本文中，我们提出了DC算法的随机变体，并给出了在几种情况下收敛到平稳点的计算复杂性。此外，我们展示了我们的方法，包括期望最大化（EM）和蒙特卡洛EM（MCEM）算法，作为训练BM的特殊情况。换句话说，我们从DC的角度出发将EM / MCEM算法扩展到更有效的方法，并且具有理论上的收敛性保证。实验结果表明，该方法在不进行预训练的情况下，对二元受限玻尔兹曼机和深部玻尔兹曼机的训练效果很好。

著录项

相似文献

  • 外文文献
  • 中文文献
  • 专利
获取原文

客服邮箱：kefu@zhangqiaokeyan.com

京公网安备：11010802029741号 ICP备案号：京ICP备15016152号-6 六维联合信息科技 (北京) 有限公司©版权所有

  • 客服微信

  • 服务号