A Fine-Grained and Dynamic MapReduce Task Scheduling Scheme for the Heterogeneous Cloud Environment
MapReduce framework is becoming more and more popular in various applications.However,Hadoop is a seriously limited by its MapReduce scheduler which does not work well in the heterogeneous environment.LATE MapReduce scheduling algorithm takes heterogeneous environment into consideration.However,it falls short of solving the poor performance due to the static manner during computing the tasks progress.In order to improve the cluster performance in a heterogeneous cloud environment,FiGMR – a Fine-Grained and dynamic MapReeduce scheduling algorithm,is proposed.FiGMR can significantly reduce the tasks execution time and improve the resources utilization.FiGMR includes historical and realtime online information obtained from each node to select the appropriate parameters to find the real slow task dynamically.Meanwhile,in order to further improve the cluster performance,FiGMR classifies map nodes into highperformance map node and low-performance map node.FiGMR classifies slow tasks into slow map tasks and slow reduce tasks.Map/Reduce slow nodes means nodes which execute map/reduce tasks using a longer time than most other nodes.In this way,FiGMR launches backup map tasks on nodes which are high-performance map nodes.
Cloud computing MapReduce scheduling Hadoop Fine-grained Heterogeneous environment
Yingchi Mao Haishi Zhong Longbao Wang
College of Computer and Information Hohai University Nanjing 211100,China
国际会议
贵阳
英文
155-158
2015-08-18（万方平台首次上网日期，不代表论文的发表时间）