• 主题
高级搜索  >
历史搜索 清空历史
首页> 外文期刊>IEEE Signal Processing Magazine >Straggler-Resistant Distributed Matrix Computation via Coding Theory: Removing a Bottleneck in Large-Scale Data Processing
【24h】

Straggler-Resistant Distributed Matrix Computation via Coding Theory: Removing a Bottleneck in Large-Scale Data Processing

机译：通过编码理论抗级别分布式矩阵计算：在大规模数据处理中删除瓶颈

获取原文
获取原文并翻译 | 示例
           
页面导航
  • 摘要
  • 著录项
  • 相似文献
  • 相关主题

摘要

The current big data era routinely requires the processing of large-scale data on massive distributed computing clusters. In these applications, data sets are often so large that they cannot be housed in the memory and/or the disk of any one computer. Thus, the data and the processing are typically distributed across multiple nodes. Distributed computation is thus a necessity rather than a luxury. The widespread use of such clusters presents several opportunities and advantages over traditional computing paradigms. However, it also presents newer challenges where coding-theoretic ideas have recently had a significant impact. Large-scale clusters (which can be heterogeneous in nature) suffer from the problem of stragglers, which are slow or failed worker nodes in the system. Thus, the overall speed of a computation is typically dominated by the slowest node in the absence of a sophisticated assignment of tasks to the worker nodes.
机译：目前的大数据时代通常需要在大规模分布式计算集群上处理大规模数据。在这些应用中，数据集通常如此大，因为它们不能容纳在任何一台计算机的存储器和/或磁盘中。因此，数据和处理通常跨多个节点分布。因此，分布式计算是必要的而不是奢侈品。这种集群的广泛使用呈现了传统计算范例的几种机会和优势。然而，它还提出了较新的挑战，其中编码理论最近产生了重大影响。大规模集群（其在自然中可能是异质的）遭受陷入困境的问题，这些问题是系统中的工人节点缓慢或失败。因此，计算的整体速度通常由最慢的节点主导，在没有对工作者节点的复杂任务分配的情况下的最慢的节点。

著录项

相似文献

  • 外文文献
  • 中文文献
  • 专利
获取原文

客服邮箱：kefu@zhangqiaokeyan.com

京公网安备：11010802029741号 ICP备案号：京ICP备15016152号-6 六维联合信息科技 (北京) 有限公司©版权所有

  • 客服微信

  • 服务号