Abstract:

Object based parallel file systems are widely used in current supercomputers for its excellent performance of large file at the expense of its lower efficiency on small file access. Through analysis of the overhead of small file access on typical object based parallel file system, a serverside small file aware cache is proposed in order to reduce file access latency by aggressively prefetching a set of related files object and improving small file hit rate. This method is implemented and evaluated in Lustrefile system which is a popular parallel file system for HPC system. The experiments in real system show that it can improve the performance of small file access dramatically, especially with the reduction of up to 88% of total cost in scenario of file archiving operation.

Key words: parallel file system;cache;object storage;prefetching