Abstract:

According to the characteristics of the 21CMA correlator algorithm, we propose a novel highefficient method to implement this specific algorithm on the Hadoop+CUDA platform, and it outperforms the MPI alone and MPI+CUDA solutions. The proposed method improves the parallel model of the correlator. Compared to the earlier MPI solution, it greatly enhances the running performance by  utilizing the advantages of GPU for FFT processing, vector multiplication and vector addition. The Hadoop software architecture, a bigdata platform, is employed in the method by using Hadoop Streaming tool to realize parallel execution of nonJava programs running on distributed systems, and linear speedups on clusters are easily obtained. In addition， the result data and procedure logs can be flexibly managed in the parallel file system of the Hadoop HDFS, which provides a well precondition for future bigdata analysis.

Key words: Hadoop;CUDA;21CMA;correlator