MyBSP:An Iterative Processing Framework Based on the Cloud Platform for Graph Data

摘要：

　　Massive cloud-based data-intensive applications(e.g.,iterative MapReduce-based)could involve graph data processing.How to effectively analyze and process large-scale graph data is an unsolved challenging problem.We present a parallel computation framework,named MyBSP,which is inspired by Googles Pregel system.MyBSP supports and implements the Bulk Synchronous Parallel(BSP)programming model,and introduces a module of parallel execution unit to achieve iterative processing,which avoids the restart cost of computation jobs,and therefore reduces the I/O overhead(e.g.,network communication and disk access).Furthermore,we implement the MyBSP-based PageRank algorithm.Some experiments are conducted to evaluate and compare the performance and scalability of our MyBSP prototype system with MapReduce model.The experimental results show that the speedup in MyBSP compared to MapReduce is about 3.5X for the small-size graph dataset.Meanwhile,the performance improvement of MyBSP also outperforms MapReduce a factor of 2.1 when processing the large-scale dataset.

关键词： BSP model iterative processing framework cloud platform graph data processing

作者: Chao Liu Hong Yao Deze Zeng Qingzhong Liang Chengyu Hu Xuesong Yan

作者单位: School of Computer Science and Technology,Huazhong University of Science and Technology,Wuhan,China; School of Computer Science China University of Geosciences,Wuhan,China

会议类型: 国际会议

会议名称: 2014 2nd International Conference on Advanced Cloud and Big Data (CBD 2014)(2014年先进云计算和大数据国际会议)

会议地点: 安徽黄山

会议语种:英文

页码: 128-135

在线出版日期: 2014-11-20（万方平台首次上网日期，不代表论文的发表时间）