Task Scheduling for MapReduce based on Heterogeneous Networks
In this paper,the task scheduling in MapReduce is considered for geo-distributed data centers on heterogeneous networks.Job deadlines and an adaptive heartbeat are concerned for data locality.With the data locality and deadline constraints,the task scheduling in the Map phase is formulated as an Assignment Problem(AP)in each heartbeat.The mapped jobs are allocated to the most suitable data centers by the earliest completion times(including both the data transfer and processing times)in the Reduce phase.A task scheduling framework TSH is proposed,in which the scheduling sequence of jobs is determined by the job deadlines,adaptive heartbeats by the processing times of tasks,and the schedule by the Hungarian algorithm.Three heuristics(TSHC,TSHA,and TSHB)are constructed based on TSH with various heartbeat intervals.Experimental results show that TSHB outperforms the other two in e ectiveness with the least computation time.
Big data MapReduce Task scheduling Data locality Adaptive heartbeat
Jia Wang Xiaoping Li
School of Computer Science & Engineering,Southeast University,Nanjing,China
国际会议
南昌
英文
1-13
2013-09-26（万方平台首次上网日期，不代表论文的发表时间）