Building quotient cube with MapReduce in hadoop
In order to solve the problem that how to improve the efficiency of query and calculation in massive data,a method of building quotient cubes in Hadoop plateform which combined the advantage of the quotient cube and MapReduce model is proposed in this paper.At first,all cubes will be established and their aggregate value will be calculated in the Mapping stage.All the key/value pair formed in Mapping stage will be passed to Reducing stage.Equivalence partitioning will be carried out In this stage,and the minimum aggregation cube of each equivalence partitioning will be the key with its aggregate value.According to the minimum aggregation cubes,we can get the quotient cubes.In order to improve the speed of parallel computing and reduce network traffic,equivalence class division will be executed locally after the Map stage,it is named as combiner stage.In this paper,MapReduce model is used to improve the efficiency of building quotient cube because of its ability of parallel computing in a large amount of data.In addition,the experiment proved that,under certain circumstances,increasing the number of Mapper/Reducer task can reduce the building time effectively,and improve the construction efficiency.
Data warehousing, Quotient cube, Hadoop plateform, MapReduce model cloud computing Datas
Juan Zhang
Department of Computer Science and Technology, East China Normal University, Shanghai 200241, China
国际会议
太原
英文
643-646
2013-04-06（万方平台首次上网日期，不代表论文的发表时间）