MapReduce Model Implementation on MPI Platform

摘要：

　　With development of Multicore clusters the taskscheduling problem in heterogeneous cluster has become hot point of research.The method to solve this problem in Cloud computing is virtualization,which can make the heterogeneous nodes being isomorphic and then using MapReduce model for task scheduling in isomorphic nodes.But the approach has some shortcomings: virtualization itself will cause the loss of performance; and there are much more disk IOs in the MapReduce model,which can also cause performance degradation.Based on our earlier work which successfully adds fault-tolerance functions in MPI,this paper proposes a MPI based MapReduce approach which implements internodes communication with efficient MPI communication functions to achieve task scheduling on heterogeneous nodes directly by improved work pool and thread pool.By this way the load balancing can be achieved efficiency.The proposed MPI based MapReduce model can efficiently deal with a kind of data intensive as well as computation intensive problems.

关键词： MapReduce MPI task scheduling load balancing

作者: Guo Yucheng

作者单位: Computer Science Department School of Computer Science and Technology Wuhan University of Technology,Wuhan,China

会议类型: 国际会议

会议名称: The 13th International Symposium on Distributed Computing and Applications to Business,Engineering and Science(DCABES 2014)(第十三届分布式计算及其应用国际学术研讨会)

会议地点: 湖北咸宁

会议语种:英文

页码: 88-91

在线出版日期: 2014-11-24（万方平台首次上网日期，不代表论文的发表时间）