Join Optimization in the MapReduce Environment for Column-wise Data Store
The chain join processing which combines records from two or more tables sequentially has been well studied in the centralized databases. However,it has seldom been discussed in the cloud computing era, and remains imperative to be solved, especially where structured (or relational) data are stored in a column (attribute) wise fashion in distributed file systems (e.g., Google File System) over hundreds of or even thousands of commodities PCs. In this paper, we propose a novel method for chain join processing, which is one of the common primitives in the cloud era for column-wise stored data analysis. By effectively selecting the dedicated records (tuples) for the chain join based on the information exploited within bipartite join graph, communication cost for record transmission could be reduced dramatically. A bushy tree structure is deployed to regulate the chain join sequence, which further reduces the number of intermediate results generated and transmitted, and explores higher parallelism in join processing, while results in more efficient join processing. Our extensive performance study confirms the effectiveness and efficiency of our methods.
Minqi Zhou Rong Zhang Dadan Zeng Weining Qian Aoying Zhou
Software Engineering Institute, East China Normal University, Shanghai 200062, China. National Institute of Information and Communications Technology, Kyoto 619-0289, Japan.
国际会议
Sixth International Conference on Semantics,Knowledge and Grids(第六届语义、知识与网格国际会议 SKG 2010)
宁波
英文
97-104
2010-11-01（万方平台首次上网日期，不代表论文的发表时间）