Based on OpenBLAS and BLIS,  the two open source linear algebra libraries, the performance optimization of dense matrix multiplication (GEMM) operation is studied. Aiming at how to select the key block parameters of GEMM, a performance optimization model is established. An improved genetic algorithm is used to solve the above performance optimization model. The performance value of the GEMM corresponding to a certain parameter combination (individual) is taken as the fitness of the individual. The optimal combination of block parameters is found through continuous iterative selection, crossover and mutation operations in order to make the performance of GEMM optimal. Numerical experiments show that the performance of GEMM based on genetic algorithm is better than the performance under the initial block parameters, and hence the optimization is achieved.