In order to improve the accuracy of multi-focus image and infrared visible image fusion, based on the advantages of the good localization and shift invariance of the finite discrete shearlet transform, we propose a new image fusion algorithm, called FDST, which combines contrast with image gradient information correlation factor. Firstly, the FDST decomposes the registration images, and obtain the low frequency sub-band coefficients and high frequency sub-band coefficients of different scales and directions. The fusion principle of low frequency sub-band coefficients bases on the method of image gradient information correlation factor for weighting. The high frequency sub-band coefficients combine the high frequency and low frequency sub-band coefficients with contrast, and uses contrast as the criterion for choosing metric coefficients which is taken as the fusion rule. Finally, the low frequency information and high frequency information are reconstructed to obtain a fused image by the finite discrete shearlet inverse transform, and both subjective visual evaluation and objective performance assessments of the fusion results are implemented. The results indicate that the algorithm is superior to other fusion algorithms in subjective visual effects and objective evaluation.