Decentralized Stochastic Optimization and Machine Learning: A Unified Variance-Reduction Framework for Robust Performance and Fast Convergence

Xin Ran; Kar Soummya; Khan Usman A.

首页> 外文期刊>IEEE Signal Processing Magazine >Decentralized Stochastic Optimization and Machine Learning: A Unified Variance-Reduction Framework for Robust Performance and Fast Convergence

【24h】

Decentralized Stochastic Optimization and Machine Learning: A Unified Variance-Reduction Framework for Robust Performance and Fast Convergence

机译：分散的随机优化和机器学习：统一的差异减少框架，可实现鲁棒性能和快速收敛性

获取原文

获取原文并翻译 | 示例

掌桥外文数据库（机构版） >>

开具论文收录证明 >>

文献代查 >>

页面导航

摘要
著录项
相似文献
相关主题

摘要

Decentralized methods to solve finite-sum minimization problems are important in many signal processing and machine learning tasks where the data samples are distributed across a network of nodes, and raw data sharing is not permitted due to privacy and/or resource constraints. In this article, we review decentralized stochastic first-order methods and provide a unified algorithmic framework that combines variance reduction with gradient tracking to achieve robust performance and fast convergence. We provide explicit theoretical guarantees of the corresponding methods when the objective functions are smooth and strongly convex and show their applicability to nonconvex problems via numerical experiments. Throughout the article, we provide intuitive illustrations of the main technical ideas by casting appropriate tradeoffs and comparisons among the methods of interest and by highlighting applications to decentralized training of machine learning models.

机译：用于解决有限和最小化问题的分散方法在许多信号处理和机器学习任务中是重要的，其中数据样本分布在节点网络上，并且由于隐私和/或资源约束，不允许原始数据共享。在本文中，我们审查了分散的随机第一阶方法，并提供了一个统一的算法框架，将差异减少与梯度跟踪相结合，以实现强大的性能和快速收敛。当客观功能平滑且强烈凸起时，我们提供了相应方法的明确理论保证，并通过数值实验表明其对非凸起问题的适用性。在整个文章中，我们通过在兴趣方法中施加适当的权衡和比较来提供主要技术思想的直观的插图，并通过突出申请来分散培训机器学习模型。

著录项

来源
《IEEE Signal Processing Magazine》 |2020年第3期|102-113|共12页
作者
Xin Ran; Kar Soummya; Khan Usman A.;
展开▼
作者单位

Carnegie Mellon Univ Dept Elect & Comp Engn Pittsburgh PA 15213 USA;

Carnegie Mellon Univ Dept Elect & Comp Engn Pittsburgh PA 15213 USA;

Tufts Univ Elect & Comp Engn Dept Medford MA 02155 USA;

展开▼
收录信息
原文格式 PDF
正文语种 eng
中图分类
关键词

相似文献

外文文献
中文文献
专利

1. Data-driven stochastic robust optimization: General computational framework and algorithm leveraging machine learning for optimization under uncertainty in the big data era [J] . Chao Ning, Fengqi You Computers & Chemical Engineering . 2018,第MARa4期

机译：数据驱动的随机鲁棒优化：利用大数据时代不确定性进行机器学习优化的通用计算框架和算法
2. Towards More Efficient Stochastic Decentralized Learning: Faster Convergence and Sparse Communication [J] . Zebang Shen, Aryan Mokhtari, Tengfei Zhou, JMLR: Workshop and Conference Proceedings . 2018,第12期

机译：走向更有效的随机分散学习：更快的融合和稀疏的交流
3. A unified stochastic framework for robust topology optimization of continuum and truss-like structures [J] . Richardson J. N., Coelho R. Filomeno, Adriaenssens S. Engineering Optimization . 2016,第1a3期

机译：统一的随机框架，用于对连续体和类桁架结构进行可靠的拓扑优化
4. Stochastic Convex Optimization: Faster Local Growth Implies Faster Global Convergence [C] . Yi Xu, Qihang Lin, Tianbao Yang International Conference on Machine Learning . 2018

机译：随机凸优化：较快的本地增长意味着更快全球收敛
5. A unified state-space and scenario tree framework for multi-stage stochastic optimization: An application to emission-constrained hydro-thermal scheduling. [D] . Rebennack, Steffen. 2010

机译：用于多阶段随机优化的统一状态空间和情景树框架：在排放受限的水热调度中的应用。
6. Decentralized Learning Framework of Meta-Survival Analysis for Developing Robust Prognostic Signatures [O] . Yi Cui, Bailiang Li, Ruijiang Li -1

机译：元生存分析的去中心化学习框架用于开发鲁棒的预后签名
7. A Unified Algorithmic Framework for Block-Structured Optimization Involving Big Data: With applications in machine learning and signal processing [O] . Hong, Mingyi, Razaviyayn, Meisam, Luo, Zhi-Quan 2016

机译：涉及大数据的块结构优化的统一算法框架：在机器学习和信号处理中的应用

Decentralized Stochastic Optimization and Machine Learning: A Unified Variance-Reduction Framework for Robust Performance and Fast Convergence

摘要

著录项

相似文献

相关主题

期刊订阅