AMREF: An Adaptive MapReduce Framework for Real Time Applications

摘要：

This paper presents AMREF, an Adaptive MapREduce Framework designed for an effective use of computational resources in data center networks to deal with real time data intensive applications. AMREF entails its adaptivity from adaptive splitter, adaptive mappers and adaptive reducers in a stochastic control manner. We use three methods, feedback control, stochastic learning with smooth filter and kalman filter to implement the framwork. Comparison among the three methods suggests they can be effectively and efficiently used to reduce the makspan in three different real-world workload scenarios.

关键词： Adaptive mapreduce Feedback control Stochastic learning control Parallel Processing

作者: Fan Zhang Junwei Cao Xiaolong Song Hong Cai Cheng Wu

作者单位: National CIMS Engineering and Research Center, Department of Automation Tsinghua University, Beijing Research Institute of Information Technology, Tsinghua University, Beijing 100084, P. R. China;Tsing National CIMS Engineering and Research Center, Department of Automation IBM China Development Laboratory National CIMS Engineering and Research Center, Department of Automation;Tsinghua National Laboratory

会议类型: 国际会议

会议名称: The Ninth International Conference on Grid and Cloud Computing(第九届网格与云计算国际学术会议 GCC 2010)

会议地点: 南京

会议语种:英文

页码: 157-162

在线出版日期: 2010-11-01（万方平台首次上网日期，不代表论文的发表时间）