• 主题
高级搜索  >
历史搜索 清空历史
首页> 外文期刊>Computers, Materials & Continua >An Optimized Resource Scheduling Strategy for Hadoop Speculative Execution Based on Non-cooperative Game Schemes
【24h】

An Optimized Resource Scheduling Strategy for Hadoop Speculative Execution Based on Non-cooperative Game Schemes

机译：基于非合作博弈方案的Hadoop推测执行资源优化调度策略

获取原文
获取原文并翻译 | 示例
           
页面导航
  • 摘要
  • 著录项
  • 相似文献
  • 相关主题

摘要

Hadoop is a well-known parallel computing system for distributed computing and large-scale data processes. "Straggling'" tasks, however, have a serious impact on task allocation and scheduling in a Hadoop system. Speculative Execution (SE) is an efficient method of processing "Straggling" Tasks by monitoring real-time running status of tasks and then selectively backing up "Stragglers" in another node to increase the chance to complete the entire mission early. Present speculative execution strategies meet challenges on misjudgement of "Straggling" tasks and improper selection of backup nodes, which leads to inefficient implementation of speculative executive processes. This paper has proposed an Optimized Resource Scheduling strategy for Speculative Execution (ORSE) by introducing non-cooperative game schemes. The ORSE transforms the resource scheduling of backup tasks into a multi-party non-cooperative game problem, where the tasks are regarded as game participants, whilst total task execution time of the entire cluster as the utility function. In that case, the most benefit strategy can be implemented in each computing node when the game reaches a Nash equilibrium point, i.e., the final resource scheduling scheme to be obtained. The strategy has been implemented in Hadoop-2.x. Experimental results depict that the ORSE can maintain the efficiency of speculative executive processes and improve fault-tolerant and computation performance under the circumstances of Normal Load, Busy Load and Busy Load with Skewed Data.
机译：Hadoop是用于分布式计算和大规模数据处理的著名并行计算系统。但是，“拖延”任务会对Hadoop系统中的任务分配和调度产生严重影响。推测执行（SE）是一种有效的方法，可以通过监视任务的实时运行状态，然后有选择地备份另一个节点中的“流浪者”，以增加提早完成整个任务的机会，从而处理“流浪者”任务。当前的推测执行策略面临对误判任务和备份节点选择不当的挑战，这导致推测执行流程的实施效率低下。通过引入非合作博弈方案，本文提出了一种用于推测执行（ORSE）的优化资源调度策略。 ORSE将备份任务的资源调度转换为多方非合作游戏问题，在该问题中，任务被视为游戏参与者，而整个集群的总任务执行时间被视为效用函数。在那种情况下，当游戏达到纳什均衡点时，即在要获得的最终资源调度方案时，可以在每个计算节点中实施最惠益策略。该策略已在Hadoop-2.x中实现。实验结果表明，在正常负载，繁忙负载和有偏斜数据的繁忙负载情况下，ORSE可以保持推测执行流程的效率，并提高容错和计算性能。

著录项

  • 来源
    《Computers, Materials & Continua》 |2020年第2期|713-729|共17页
  • 作者

  • 作者单位

    Jiangsu Collaborative Innovation Center of Atmospheric Environment and Equipment Technology (CICAEET) Nanjing University of Information Science & Technology Nanjing 210044 China;

    School of Computer and Software Nanjing University of Information Science & Technology Nanjing 210044 China School of Computing Edinburgh Napier University Edinburgh EH 10 5DT UK;

    School of Computer and Software Nanjing University of Information Science & Technology Nanjing 210044 China;

    School of Computing Edinburgh Napier University Edinburgh EH 10 5DT UK;

    School of Electrical Engineering University of Jinan China and Centre for Health Sciences Research University of Salford Salford Greater Manchester M5 4WT UK;

  • 收录信息
  • 原文格式 PDF
  • 正文语种 eng
  • 中图分类
  • 关键词

    Distributed computing; speculative execution; resource scheduling; non-cooperative game theory;

    机译：分布式计算;投机执行;资源调度;非合作博弈论;

相似文献

  • 外文文献
  • 中文文献
  • 专利
获取原文

客服邮箱：kefu@zhangqiaokeyan.com

京公网安备：11010802029741号 ICP备案号：京ICP备15016152号-6 六维联合信息科技 (北京) 有限公司©版权所有

  • 客服微信

  • 服务号