Research of job scheduling in Hadoop, based on analysis of requirement for Hadoop job scheduling algorithm, the solution space in linear meaning of scheduling algorithm is proposed. Aiming at the procedure model for Hadoop, an improved Artificial Fish Swarm Algorithm ( AFSA) combines tabu search is put forward. It uses total execution time as the optimized functions,and with linear coding,each N-dimensional vector represents a special scheduling scheme. The method which takes solution vector as artificial fish di-rectly is applied to make AFSA can be run directly in the solution space. IAFSA not only retains the advantages of rapid convergence of AFSA at a large computing base,also makes full use of the advantages of tabu search does not fall into local optima. Through comparison between the algorithm with the Fair algorithm,the experiment shows that the improved AFSA in heterogeneous environments can improve system performance and reduce the computing time. It is effective in the Hadoop environment.%对Hadoop中的任务调度进行了研究,在分析Hadoop作业调度算法的需求的基础上,文中提出了调度算法在线性意义上的解空间.针对Hadoop的编程模型框架,提出了一种结合禁忌搜索思想的改进人工鱼群算法.在该算法中,以任务总执行时间作为寻优函数,采用线性编码方式,每一个N维向量代表一种具体调度方案;利用将解向量直接作为人工鱼的方法,使人工鱼群算法可以直接在解空间内运行.结合禁忌搜索思想,既保留了人工鱼群算法计算基数大仍能快速收敛的优点,又充分利用禁忌搜索不会陷入局部最优解的优势.通过仿真实验将该算法和Fair算法进行比较,结果表明:改进的人工鱼群作业调度算法可以提高系统性能,降低任务运行时间,是一种Hadoop环境下有效的任务调度程序.
展开▼
