Research on Optimization of Association Rules Algorithm Based on Spark

摘要：

　　Aiming at the bottleneck of traditional association rule algorithm(Apriori),such as processing speed and computing resources,as well as the problem of accessing disk in the MapReduce computing framework on Hadoop platform.The traditional association rules are transferred to the memory based Spark computing framework,and the optimization algorithm under the framework of Spark is given.By comparing the Apriori algorithm under MapReduce,the algorithm can greatly improve the mining efficiency of the large data association rules.At the same time,the algorithm can reduce the I/O overhead when facing a large number of data.In the cluster,both the extensibility and the acceleration ratio are better than the traditional Apriori algorithm.

关键词： Association rules Aprior Spark pruning

作者: Chengang Li Yu Liu Zeng Li

作者单位: College of Information Science and Engineering,Guilin University of Technology,Guilin,China

会议类型: 国际会议

会议名称: 2019 2nd International Conference on Mechanical, Electronic and Engineering Technology (MEET 2019) 2019年第二届机电与工程技术国际会议

会议地点: 西安

会议语种:英文

页码: 460-465

在线出版日期: 2019-01-19（万方平台首次上网日期，不代表论文的发表时间）