Performance Optimization of Small File I/O with Adaptive Migration Strategy in Cluster File System
While cluster file systems exploit data striping scheme to boost large file I/O throughput, small file performance is impaired and neglected. Common metadata-based optimizations introduce obstacles such as metadata server overload and migration latency. In this paper, a novel adaptive migration strategy is incorporated into metadata-based optimization to alleviate these side effects by migrating file dynamically. Guided by proposed adaptive migration threshold model, two types of file migration are applied to reduce metadata server load without degrading current performance of file systeM obviously. Schemes of latency hiding and migration consistency are also introduced to reduce overhead induced by small file optimization. Our results indicate that proposed optimization can substantially improve file creation and deletion performance, and boost small file I/O throughput by more than 20%. Moreover, side effects on overall performance produced by file migration are slight and can be absorbed by improvements.
Xiuqiao Li Bin Dong Limin Xiao Li Ruan
School of Computer Science and Engineering BeiHang University 100191 Beijing, P.R. China
国际会议
The Second International Conference on High Performance Computing and Applications(第二届高性能计算及应用国际会议)
上海
英文
242-249
2009-08-10（万方平台首次上网日期，不代表论文的发表时间）