• 主题
高级搜索  >
历史搜索 清空历史
首页> 外文期刊>IEEE Transactions on Parallel and Distributed Systems >Hybrid Job-Driven Scheduling for Virtual MapReduce Clusters
【24h】

Hybrid Job-Driven Scheduling for Virtual MapReduce Clusters

机译：虚拟MapReduce群集的混合作业驱动调度

获取原文
获取原文并翻译 | 示例
       
页面导航
  • 摘要
  • 著录项
  • 相似文献
  • 相关主题

摘要

It is cost-efficient for a tenant with a limited budget to establish a virtual MapReduce cluster by renting multiple virtual private servers (VPSs) from a VPS provider. To provide an appropriate scheduling scheme for this type of computing environment, we propose in this paper a hybrid job-driven scheduling scheme (JoSS for short) from a tenant's perspective. JoSS provides not only job-level scheduling, but also map-task level scheduling and reduce-task level scheduling. JoSS classifies MapReduce jobs based on job scale and job type and designs an appropriate scheduling policy to schedule each class of jobs. The goal is to improve data locality for both map tasks and reduce tasks, avoid job starvation, and improve job execution performance. Two variations of JoSS are further introduced to separately achieve a better map-data locality and a faster task assignment. We conduct extensive experiments to evaluate and compare the two variations with current scheduling algorithms supported by Hadoop. The results show that the two variations outperform the other tested algorithms in terms of map-data locality, reduce-data locality, and network overhead without incurring significant overhead. In addition, the two variations are separately suitable for different MapReduce-workload scenarios and provide the best job performance among all tested algorithms.
机译：对于预算有限的租户而言，通过从VPS提供商那里租用多个虚拟专用服务器（VPS）来建立虚拟MapReduce集群具有成本效益。为了为这种类型的计算环境提供合适的调度方案，我们从租户的角度提出了一种混合作业驱动的调度方案（简称JoSS）。 JoSS不仅提供作业级调度，还提供地图任务级调度和缩减任务级调度。 JoSS根据作业规模和作业类型对MapReduce作业进行分类，并设计适当的调度策略来调度每类作业。目标是改善两个地图任务的数据局部性并减少任务，避免作业匮乏并提高作业执行性能。进一步引入了JoSS的两种变体，以分别实现更好的地图数据局部性和更快的任务分配。我们进行了广泛的实验，以使用Hadoop支持的当前调度算法评估和比较这两种变体。结果表明，这两种变体在地图数据局部性，减少数据局部性和网络开销方面均优于其他测试算法，而不会产生大量开销。此外，这两种变体分别适用于不同的MapReduce工作负载场景，并在所有经过测试的算法中提供最佳的工作性能。

著录项

相似文献

  • 外文文献
  • 中文文献
  • 专利
获取原文

客服邮箱：kefu@zhangqiaokeyan.com

京公网安备：11010802029741号 ICP备案号：京ICP备15016152号-6 六维联合信息科技 (北京) 有限公司©版权所有

  • 客服微信

  • 服务号