STUDY ON THE SMALL FILES PROBLEM OF HADOOP

摘要：

　　Although Hadoop is widely used,its full potential is not yet put to use because of some issues,the small files problem being one of them.Firstly,the paper analyses the causes of the small files problem of Hadoop.Then,the current program to solve the small files problem are introduced,including Hadoop own programs and other application-specific solutions,and analyzes the advantages and disadvantages of various options.Finally,we present two research ideas,one is to use a combination of RDBMS and Hadoop; Another is to make the Datanode caching some metadata of the small files.

关键词： Hadoop Distributed File System (HDFS) Small Files Problem Hadoop Archives Sequence files RDBMS

作者: Xiaojun Liu Zhengquan Xu Xin Gu

作者单位: LIESMARS,Wuhan University,Wuhan 430079,China;Huanggang Normal University,Huanggang 438000,China LIESMARS,Wuhan University,Wuhan 430079,China

会议类型: 国际会议

会议名称: 2012 2nd IEEE International Conference on Cloud Computing and Intelligence Systems (2012年第2届IEEE云计算与智能系统国际会议(IEEE CCIS2012))

会议地点: 杭州

会议语种:英文

页码: 278-281

在线出版日期: 2012-10-30（万方平台首次上网日期，不代表论文的发表时间）