Abstract:

MapReduce has become a mainstream mass data processing mode,As its crucial part, the scheduler has received extensive concerns of the industry.But existing scheduling algorithms cannothavea good balance between fairness and data locality.Therefore,in this paper, a selfadaptive delay scheduling algorithm is proposed in order to make up the shortage of statically setting waiting time in configuration file.It dynamically adjust waiting time according to the speed of free nodes,so it can reduce job response time.Based on the algorithm,a prototype is developed and experiments testing the algorithm's performance are carried out.The results show that the selfadaptive delay schedulingalgorithm outperforms previous delay scheduling ones in term of the job response time about 5%~8%.

Key words: fair scheduler;selfadaptive;delay scheduling;data locality