Balanced Parallel FP-Growth with MapReduce

摘要：

Frequent itemset mining (FIM) plays an essential role in mining associations, correlations and many other important data mining tasks. Unfortunately, as the volume of dataset gets larger day by day, most of the FIM algorithms in literature become ineffective due to either too huge resource requirement or too much communication cost. In this paper, we propose a balanced parallel FP-Growth algorithm BPFP, based on the PFP algorithm 1, which parallelizes FP-Growth in the MapReduce approach. BPFP adds into PFP load balance feature, which improves parallelization and thereby improves performance. Through empirical study, BPFP outperformed the PFP which uses some simple grouping strategy.

关键词： Algorithms Distributed computing

作者: Le Zhou Zhiyong Zhong Jin Chang Junjie Li Joshua Zhexue Huang Shengzhong Feng

作者单位: Center for High Performance Computing Institute of Advanced Computing and Digital Engineering Shenzhen Institutes of Advanced Technologies, Chinese Academy of Sciences

会议类型: 国际会议

会议名称: 2010 IEEE Youth Conference on Information,Computing and Telecommunications(2010 IEEE青年信息、计算和通信技术大会)

会议地点: 北京

会议语种:英文

页码: 243-246

在线出版日期: 2010-11-28（万方平台首次上网日期，不代表论文的发表时间）