对象存储并行文件系统小文件性能优化研究

J4 ›› 2013, Vol. 35 ›› Issue (12): 8-13.

对象存储并行文件系统小文件性能优化研究

周恩强，董勇，张伟，卢宇彤

（国防科学技术大学高性能计算国家重点实验室,湖南长沙 410073）

收稿日期:2013-08-11 修回日期:2013-10-15 出版日期:2013-12-25 发布日期:2013-12-25
基金资助:
国家自然科学基金资助项目（61120106005）；国家863计划资助项目（2012AA01A301）

Optimization of small file access on
object based parallel file system

ZHOU Enqiang，DONG Yong，ZHANG Wei，LU Yutong

(State Key Laboratory of High Performance Computing,National University of Defense Technology,Changsha 410073,China)

Received:2013-08-11 Revised:2013-10-15 Online:2013-12-25 Published:2013-12-25

摘要/Abstract

摘要：

对象存储并行文件系统在高性能计算领域被广泛采用，虽然大文件访问性能优异，但其小文件访问存在局限性。通过分析典型对象存储并行文件系统下的小文件的访问开销，采用服务端小文件专用缓存和基于对象命名关联性预取的方法提高小文件的缓存命中率，降低对象存储架构下小文件的读取延迟。基于Lustre并行文件系统实现了该方法。实际系统测试表明，该方法可显著提高小文件访问性能，其中小文件打包性能提升最大可达88%。

关键词: 并行文件系统, 缓存, 对象存储, 文件预取

Abstract:

Object based parallel file systems are widely used in current supercomputers for its excellent performance of large file at the expense of its lower efficiency on small file access. Through analysis of the overhead of small file access on typical object based parallel file system, a serverside small file aware cache is proposed in order to reduce file access latency by aggressively prefetching a set of related files object and improving small file hit rate. This method is implemented and evaluated in Lustrefile system which is a popular parallel file system for HPC system. The experiments in real system show that it can improve the performance of small file access dramatically, especially with the reduction of up to 88% of total cost in scenario of file archiving operation.

Key words: parallel file system;cache;object storage;prefetching

周恩强，董勇，张伟，卢宇彤. 对象存储并行文件系统小文件性能优化研究[J]. J4, 2013, 35(12): 8-13.

ZHOU Enqiang，DONG Yong，ZHANG Wei，LU Yutong. Optimization of small file access on
object based parallel file system [J]. J4, 2013, 35(12): 8-13.

参考文献

［1］Object storage architecture. Panasas white paper［EB/OL］. ［20090101］. http:∥

www.panasas.com.
［2］Lustre file system ［EB/OL］.［20050807］. http:∥www.lustre.org.
［3］Cho H,Kim S,Lee S.Analysis of longterm file system activities on cluster systems［J］

. World Academy of Science Engineering and Technology, 2009, 60(1):159164.
［4］Hendricks J, Sambasivan R R, Sinnamohideen S, et al. Improving small file performance

in objectbased storage［R］. CMUPDL06104,Pittsburgh:Carnegie Mellon University,

2006.
［5］Dahlin M D, Wang R Y, Anderson T E, et al. Cooperative caching:Using remote client

memory to improve file system performance［C］∥Proc of the 1st Symposium on Operating

Systems Design and Implementation, 1994:1.
［6］Lu kai, Jin Shiyao, Lu Xicheng. Properly greedy cache prefetch integrated algorithm

in the parallel file system［J］. Chinese Journal of Computers, 1999, 22(11):11721177. (in

Chinese)
［7］Patil S V, Gibson G, Lang S, et al. Giga+:Scalable directories for shared file systems

［C］∥Proc of the 2nd International Workshop on Petascale Data Storage, 2007:2629.
［8］Dong Yong, Zhou Enqiang, Chen Juan. Infinibandbased highperformance distributed

file systemLustre［J］. Computer Engineering and Applications, 2005, 41(22):103107. (in

Chinese)
［9］Polte M, Simsa J, Gibson G. Comparing performance of solid state devices and mechanical

disks［C］∥Proc of the 3rd Petascale Data Storage Workshop, 2008:17.
［10］Lin Lin, Li Xuemin, Jiang Hong, et al. AMP:An affinitybased metadata prefetching

scheme in largescale distributed storage systems［C］∥Proc of Cluster of Computing and

the Grid,2008:459466.
附中文参考文献：
［6］卢凯，金士尧，卢锡城.并行文件系统中适度贪婪的cache预取一体化算法［J］.计算机学报，1999

，22（11）：11721177.
［8］董勇，周恩强，陈娟.基于Infiniband技术构建高性能分布式文件系统Lustre［J］.计算机工程与

应用，2005，41（22）：103107.

[1]	王炜琛, 涂海洋, 王伟明, 赵晓博. 基于Zynq的SDI视频图像分割系统[J]. 计算机工程与科学, 2021, 43(10): 1796-1802.
[2]	李仁刚, 任智新, 黄广奎, 孙颉, 王峰, 张闯, . 面向数据库查询加速的异构体系结构设计与实现[J]. 计算机工程与科学, 2020, 42(12): 2169-2178.
[3]	宋振龙, 李小芳, 李琼, 谢徐超, 魏登萍, 董勇, 王睿伯. BeeGFS并行文件系统性能优化技术研究[J]. 计算机工程与科学, 2020, 42(10高性能专刊): 1765-1773.
[4]	贾朝阳, 张敦博, 王琼, 沈立. 一种高效的压缩Page Walk Cache结构[J]. 计算机工程与科学, 2020, 42(09): 1521-1528.
[5]	张建伟1，王旭辉2，蔡增玉2，黄万伟1，杜春锋2. 基于势能冷却的内容中心网络缓存替换算法[J]. 计算机工程与科学, 2019, 41(09): 1612-1617.
[6]	文凯1,2,3,谭笑1,2. D2D网络中的缓存副本数布设算法[J]. 计算机工程与科学, 2019, 41(08): 1421-1425.
[7]	方娟，魏泽琳，于婷雯. 一种基于Inter-warp异构性的缓存管理与内存调度机制[J]. 计算机工程与科学, 2019, 41(05): 788-795.
[8]	于美菊，李茹. 基于动态内容流行度的NDN缓存决策和替换策略研究[J]. 计算机工程与科学, 2019, 41(02): 275-280.
[9]	曹立强,罗红兵. 并行科学计算应用中采样数据的聚集I/O[J]. 计算机工程与科学, 2018, 40(09): 1534-1539.
[10]	侯永宏，邢家明，王利伟. 基于模糊控制的流媒体自适应传输算法[J]. 计算机工程与科学, 2018, 40(08): 1444-1452.
[11]	陈逸飞，李宏亮，刘骁，高红光. 一种阵列众核处理器的多级指令缓存结构[J]. 计算机工程与科学, 2018, 40(04): 571-579.
[12]	张昆,刘骁,郑方,谢向辉. 众核处理器的共享一级指令缓存研究[J]. 计算机工程与科学, 2017, 39(05): 834-840.
[13]	张昆,郝子宇，郑方，谢向辉. 零级指令缓存研究综述[J]. 计算机工程与科学, 2017, 39(03): 405-412.
[14]	刘骁，高红光，陈芳园，丁亚军. 一种高能效的结构不对称指令缓存[J]. 计算机工程与科学, 2017, 39(03): 443-450.
[15]	杨文祥，董德尊，李存禄，雷斐，孙凯旋，吴际. 一种多级无缓存高阶路由器的设计与实现[J]. 计算机工程与科学, 2017, 39(02): 245-251.