• 主题
高级搜索  >
历史搜索 清空历史
首页> 外文会议>IEEE International Congress on Big Data >Configuring a MapReduce Framework for Performance-Heterogeneous Clusters
【24h】

Configuring a MapReduce Framework for Performance-Heterogeneous Clusters

机译：为性能异构集群配置MapReduce框架

获取原文
页面导航
  • 摘要
  • 著录项
  • 相似文献
  • 相关主题

摘要

When data centers employ the common and economical practice of upgrading subsets of nodes incrementally, rather than replacing or upgrading all nodes at once, they end up with clusters whose nodes have non-uniform processing capability, which we also call performance-heterogeneity. Popular frameworks supporting the effective MapReduce programming model for Big Data applications do not flexibly adapt to these environments. Instead, existing MapReduce frameworks, including Hadoop, typically divide data evenly among worker nodes, thereby inducing the well-known problem of stragglers on slower nodes. Our alternative MapReduce framework, called MARLA, divides each worker's labor into sub-tasks, delays the binding of data to worker processes, and thereby enables applications to run faster in performance-heterogeneous environments. This approach does introduce overhead, however. We explore and characterize the opportunity for performance gains, and identify when the benefits outweigh the costs. Our results suggest that frameworks should support finer grained sub-tasking and dynamic data partitioning when running on some performance-heterogeneous clusters. Blindly taking this approach in homogeneous clusters can slow applications down. Our study further suggests the opportunity for cluster managers to build performance-heterogeneous clusters by design, if they also run MapReduce frameworks that can exploit them.
机译：当数据中心采用逐步升级节点子集的公共和经济实践时，而不是一次替换或升级所有节点，它们最终包含群集，其节点具有非均匀处理能力，我们还调用了性能异质性。支持大数据应用的有效MapReduce编程模型的流行框架不会灵活地适应这些环境。相反，现有的MapReduce框架（包括Hadoop）通常在工作节点中均匀地划分数据，从而在较慢的节点上诱导孤立者的众所周知的问题。我们的替代MapReduce框架称为Marla，将每个工人的劳动力分成子任务，将数据的绑定延迟到工人进程，从而使应用程序能够在性能异构环境中更快地运行。然而，这种方法会引入开销。我们探索并表征绩效收益的机会，并确定何时何时兴趣超过成本。我们的结果表明，在某些性能异构集群上运行时，框架应支持更精细的粒度子任务和动态数据分区。盲目地在均匀集群中采取这种方法可以缓慢应用。我们的研究进一步暗示集群管理人员通过设计构建性能 - 异构集群，如果他们还运行可以利用它们的MapReduce框架。

著录项

相似文献

  • 外文文献
  • 中文文献
  • 专利
获取原文

客服邮箱：kefu@zhangqiaokeyan.com

京公网安备：11010802029741号 ICP备案号：京ICP备15016152号-6 六维联合信息科技 (北京) 有限公司©版权所有

  • 客服微信

  • 服务号