• 中国计算机学会会刊
  • 中国科技核心期刊
  • 中文核心期刊

Computer Engineering & Science ›› 2022, Vol. 44 ›› Issue (04): 571-583.

• High Performance Computing • Previous Articles     Next Articles

consensus algorithms for distributed storage system

SHEN Jia-jie1,LU Xiu-wen1,2,3,XIANG Wang1,ZHAO Ze-yu1,WANG Xin1,2,3   

  1. (1.Informatization Office,Fudan University,Shanghai 200433;
    2.School of Computer Science,Fudan University,Shanghai 200433;
    3.Shanghai Key Laboratory of Intelligent Information Processing,Fudan University,Shanghai 200433,China)
  • Received:2021-10-08 Revised:2021-12-03 Accepted:2022-04-25 Online:2022-04-25 Published:2022-04-20

Abstract: Consensus algorithms are widely adopted in the distributed storage systems to ensure the correctness of the I/O operations. Since consensus algorithms typically use a complex protocol to ensure the correctness of the I/O operations between multiple storage nodes, it incurs high network transmission overhead and I/O delay. Due to the large differences in the implementation mechanisms of various consensus algorithms, specific consensus algorithms often need to be deployed in specific storage application scenarios in order to efficiently perform I/O operations and ensure the quality of service of the applications on them. Therefore, in the actual storage system development process, developers often need to select consensus algorithms according to storage application scenarios, thereby reducing the system overhead caused by I/O operations. In order to clarify the suitable application scenarios of various consensus algorithms, this paper introduces the consensus problems existing in distributed storage systems, and summarizes the implementation mechanism of current consensus algorithms. This paper summarizes the mainstream consensus algorithms under the replica-based storage systems and the erasure-coded storage systems, and compares the characteristics of these consensus algorithms in terms of implementation mechanism, network overhead, and data storage overhead. On this basis, this paper combines two classic application scenarios of a single data center distributed storage system and a cross-data center cloud-to-cloud storage system, and summarizes the main points that developers need to pay attention to when deploying consensus algorithms in actual storage systems. The problems that need to be solved urgently and the possible ways to improve the performance of I/O operations are analyzed, and the future development direction of the consensus algorithm is prospected. 

Key words: consensus algorithm, distributed storage system, erasure-coded storage system, I/O operation, performance optimization