Comparing Hadoop and Fat-Btree Based Access Method for Small File I/O Applications
Hadoop has been widely used in various clusters to build scalable and high performance distributed file systems. However, Hadoop distributed file system (HDFS) is designed for large file management. In case of small files applications, those metadata requests will flood the network and consume most of the memory in Namenode thus sharply hinders its performance. Therefore, many web applications do not benefit from clusters with centered metanode, like Hadoop. In this paper, we compare our Fat-Btree based data access method, which excludes center node in clusters, with Hadoop. We show their different performance in different file I/O applications.
Parallel Database Fat-Btree Hadoop File I/O
Min Luo Haruo Yokota
Department of Computer Science, Tokyo Institute of Technology 2-12-1 Ookayama, Meguro-ku, Tokyo 152- Global Scientific Information and Computing Center, Tokyo Instititute of Technology 2-12-10okayama,
国际会议
11th International Conference,WAIM 2010(第十一届网络时代管理国际会议)
九寨沟
英文
182-193
2010-07-14（万方平台首次上网日期，不代表论文的发表时间）