This paper presents a finegrained pipeline parallel algorithm for the Cholesky decomposition, which is applicable to the matrices of arbitrary orders and can exploit finegrained parallelism of the FPGA accelerators. The experimental results show this algorithm has good scalability. 36 processing elements (PEs) can be integrated into a Xilinx XC5VLX330 FPGA, achieving a performance of 14.3 Gflops when the matrix order is 16 384 at the clock speed of 200 MHz.