首页> 中文期刊> 《现代电子技术》 >Hadoop集群作业调度算法优化技术研究

Hadoop集群作业调度算法优化技术研究

         

摘要

Conception of studying Hadoop cluster job scheduling algorithm is composed in allusion to extensive application of the current cloud computing technology. According to the deep study of Hadoop Map Reduce V2(Yarn)in the new edition Hadoop,a complete contrast experiment was designed,and the advantages and disadvantages of all kinds of job scheduling algo⁃rithms were tested. The comparison was carefully made by applying the different job scheduling algorithms on the same job. The differences and characters in calculation ability,operation time,and resource occupation of all the job scheduling algorithms and calculation frameworks were obtained. The experimental results show that the fair scheduling algorithm and the calculation capacity scheduling algorithm are more flexible and more efficient than the traditional FIFO scheduling algorithm.%针对当前云计算技术的广泛使用,提出对Hadoop集群作业调度算法进行研究的构想。在对Hadoop新版本中提出的Hadoop Map Reduce V2(Yarn)框架进行深入研究的基础上,设计一个详细完整的对比实验,针对各种作业调度算法的优势与不足进行全面的测试。通过使用不同的作业调度算法运行相同的作业,进行细致横向的对比,并得到各种作业调度算法之间以及计算框架之间在计算能力、运行时间、资源占用等方面的优劣。实验结果表明,公平调度算法和计算能力调度算法相较于传统的FIFO算法具有更灵活、更高效的特点。

著录项

相似文献

  • 中文文献
  • 外文文献
  • 专利
获取原文

客服邮箱:kefu@zhangqiaokeyan.com

京公网安备:11010802029741号 ICP备案号:京ICP备15016152号-6 六维联合信息科技 (北京) 有限公司©版权所有
  • 客服微信

  • 服务号