• 主题
高级搜索  >
历史搜索 清空历史
首页> 外文期刊>IEEE Transactions on Parallel and Distributed Systems >Distributed Training of Deep Learning Models: A Taxonomic Perspective
【24h】

Distributed Training of Deep Learning Models: A Taxonomic Perspective

机译：深度学习模型的分布式培训：分类视角

获取原文
获取原文并翻译 | 示例
           
页面导航
  • 摘要
  • 著录项
  • 相似文献
  • 相关主题

摘要

Distributed deep learning systems (DDLS) train deep neural network models by utilizing the distributed resources of a cluster. Developers of DDLS are required to make many decisions to process their particular workloads in their chosen environment efficiently. The advent of GPU-based deep learning, the ever-increasing size of datasets, and deep neural network models, in combination with the bandwidth constraints that exist in cluster environments require developers of DDLS to be innovative in order to train high-quality models quickly. Comparing DDLS side-by-side is difficult due to their extensive feature lists and architectural deviations. We aim to shine some light on the fundamental principles that are at work when training deep neural networks in a cluster of independent machines by analyzing the general properties associated with training deep learning models and how such workloads can be distributed in a cluster to achieve collaborative model training. Thereby we provide an overview of the different techniques that are used by contemporary DDLS and discuss their influence and implications on the training process. To conceptualize and compare DDLS, we group different techniques into categories, thus establishing a taxonomy of distributed deep learning systems.
机译：分布式深度学习系统（DDLS）通过利用群集的分布式资源来列车深度神经网络模型。 DDL的开发人员需要多项决策，以有效地处理其所选环境中的特定工作负载。基于GPU的深度学习的出现，数据集的不断增加的大小和深度神经网络模型，与集群环境中存在的带宽约束相结合，需要DDL的开发人员创新，以便快速训练高质量模型。由于其广泛的特征列表和架构偏差，并排将DDL并排困难。我们的目标是通过分析与培训深度学习模型相关的一系列独立机器中的一群独立机器中的深度神经网络，以及如何在集群中分发，以实现协作模型的一般性来发光训练。因此，我们提供当代DDLS使用的不同技术概述，并讨论其对培训过程的影响和影响。要概念化和比较DDLS，我们将不同的技术分类为类别，从而建立了分布式深度学习系统的分类。

著录项

相似文献

  • 外文文献
  • 中文文献
  • 专利
获取原文

客服邮箱：kefu@zhangqiaokeyan.com

京公网安备：11010802029741号 ICP备案号：京ICP备15016152号-6 六维联合信息科技 (北京) 有限公司©版权所有

  • 客服微信

  • 服务号