Journal of Jilin University(Information Science Ed

Previous Articles     Next Articles

Hadoop YARN Scheduling Algorithm Based on Priority Weight

WANG Rongli, HOU Xiuping   

  1. School of Computer Science and Engineering, Changchun University of Technology, Changchun 130012, China
  • Received:2016-07-28 Online:2017-09-29 Published:2017-10-23

Abstract: In order to solve the problem that the existing scheduling task scheduler Hadoop can not depend on
the degree of urgency task allocation of resources. We mainly research the resource scheduling mechanism in
YARN (Yet Another Resource Negotiator), improved the original capacity scheduler and proposes abased on
priority weighted Hadoop YARN scheduling algorithm-weight scheduler based on priority. Firstly, setting queue
priority for leaf queue, combining with the queue resource utilization and queue priority to select queue;
Secondly, the initial of applications re-set its priorities size, by waitting time to judge whether to update the
weight, according to the weight to sort the applications in the selected queue. The experiment shows that
comparing with the existing scheduling algorithm, this algorithm has the advantage on the execution time by
141 s and waiting time by 34. 5% decreasing, ensure the user to perform a task relatively fair, improve the
overall satisfaction of users.

Key words:  Hadoop yet another resource negotiator(YARN), resource scheduler, priority weight

CLC Number: 

  • TP391