J4 ›› 2013, Vol. 51 ›› Issue (01): 101-106.

• 计算机科学 • 上一篇    下一篇

Hadoop平台下基于资源预测的Delay调度算法

魏晓辉, 付庆午, 李洪亮   

  1. 吉林大学 计算机科学与技术学院, 长春 130012
  • 收稿日期:2012-01-19 出版日期:2013-01-26 发布日期:2013-01-31
  • 通讯作者: 李洪亮 E-mail:hongliang09@mails.jlu.edu.cn

Resource Forecast Delay Algorithm for Hadoop Systems

WEI Xiaohui, FU Qingwu, LI Hongliang   

  1. College of Computer Science and Technology, Jilin University, Changchun 130012, China
  • Received:2012-01-19 Online:2013-01-26 Published:2013-01-31
  • Contact: LI Hongliang E-mail:hongliang09@mails.jlu.edu.cn

摘要:

针对Delay算法的不合理等待问题, 提出一种基于资源预测的Delay调度算法(RFD), 该算法基于对资源可用性的预测方法合理地调度作业. 实验结果表明, 在Hadoop机群一般应用场景下, 该调度算法与已有算法相比, 在保证作业本地化计算Map任务比例相近的同时, 将作业平均运行效率提高28.8%, 明显提高了Mapreduce作业的执行效率.

关键词: Hadoop平台, Mapreduce模型, 资源调度, 资源预测, Delay调度

Abstract:

In order to solve the unreasonable wait existed in delay algorithm, this paper presents a method based on resource forecast delay scheduling algorithm, by which one reasonably dispatches operation  on the basis of resource availability prediction method. The experimental results show that in the Hadoop cluster general application scenario, the scheduling algorithm, compared with related work, makes the average operating efficiency increased by a factor of 288%  in the assurance of localization computing map task to be similar. This work will greatly improve the working efficiency of Mapreduce.

Key words: Hadoop platform, Mapreduce model, resource scheduling, resource forecast, delay scheduling

中图分类号: 

  • TP316.4