Abstract:

As an open source platform of cloud computing, Hadoop is widely used in many fields, such as natural language processing, machine learning and largescale image processing. With the increase of the types of cloud services, the realtime requirement is strengthened by cloud users. Most existing schedulers are designed to shorten the response time which cannot guarantee a specific deadline. Least Sparetime Scheduler (LSS) is designed and implemented to improve the performance of hard realtime jobs in Hadoop. The spare time is estimated dynamically and the LSS updates the job priority of the job queue in realtime. Experimental results show that the LSS can improve the success ratio of the cluster dealing with hard realtime jobs.

Key words: Hadoop;realtime jobs;scheduling;spare time