Accelerate MapReduce on GPUs with Multi-level Reduction
With Graphics Processing Units (GPUs) becoming more and more popular in general purpose computing,more attentions have been paid on building a framework to provide convenient interfaces for GPU programming.MapReduce can greatly simplify the programming for data-parallel applications in cloud computing environment,and it is also naturally suitable for GPUs.However,there are some problems in recent reduction-based MapReduce implementation on GPUs.Its performance is dramatically degraded when handling massive distinct keys because the massive data cannot be stored in tiny shared memory entirely.A new MapReduce framework on GPUs,called Jupiter,is proposed with continuous reduction structure.Two improvements are supported in Jupiter,a multi-level reduction scheme tailored for GPU memory hierarchy and a frequencybased cache policy on key-value pairs in shared memory.Shared memories are utilized efficiently for various dataparallel applications whether involving little or abundant distinct keys.Experiments show that Jupiter can achieve up to 3x speedup over the original reduction-based GPU MapReduce framework on the applications with lots of distinct keys.
GPU Programming MapReduce Multi-level Reduction
Ran Zheng Kai Liu Hai Jin Qin Zhang Xiaowen Feng
Services Computing Technology and System Lab Cluster and Grid Computing Lab School of Computer Science and TechnologyHuazhong University of Science and Technology,Wuhan,430074,China
国际会议
长沙
英文
97-104
2013-10-23（万方平台首次上网日期，不代表论文的发表时间）