关键词: 海量小文件, 检索, 倒排索引, 动态重构

Abstract:

The small files distributed file system (SMDFS) can efficiently manage ten billions of files. However, a huge amount of data such as photos, music, etc., often needs to quickly browse files from multiple dimensions, and traditional files organization schemes based on the directory structure to manage massive files cannot  easily meet this requirement. Based on the SMDFS file system, we introduce features to file attributes and put forward a featurebased massive small files inverted indexing technique and a distributed indexing technique, which enables the SMDFS browse files based on multiple features. Experimental results show that the featuresupported SMDFS can provide efficient management and rapid multidimensional browsing capability for massive small files while the massive small files access performance based on filedirectory structure is not significantly decreased.

Key words: massive small files, search, inverted index, dynamic reconstruction