• 中国计算机学会会刊
  • 中国科技核心期刊
  • 中文核心期刊

Computer Engineering & Science ›› 2020, Vol. 42 ›› Issue (09): 1578-1586.

Previous Articles     Next Articles

A high spatial temporal fusion method based on deep learning and super resolution reconstruction

ZHANG Yong-mei1,HUA Rui-min1,MA Jian-zhe2,HU Lei3   

  1. (1.School of Information Science and Technology,North China University of Technology,Beijing 100144;

     2.Department of Electronic & Information Engineering,The Hong Kong Polytechnic University,Hong Kong 00852;

    3.School of Computer Information Engineering,Jiangxi Normal University,Nanchang 330022,China)

  • Received:2019-11-07 Revised:2020-02-22 Accepted:2020-09-25 Online:2020-09-25 Published:2020-09-24

Abstract: Aiming at the "space-time conflict" of remote sensing images, a high spatial-temporal fusion algorithm based on improved STARFM is proposed. SRCNN is used for the super-resolution reconstruction of low-resolution images. Due to the large difference in resolution between the two groups of fusion images, the network training is difficult. Firstly, both of the two groups are sampled to an intermediate resolution, and low-resolution images are reconstructed by SRCNN with high-resolution images as their prior knowledge. Secondly, the obtained intermediate resolution images are resampled, and then they are reconstructed by SRCNN with original high-resolution images as their prior knowledge. The resulting reconstructed images have higher PSNR and SSIM than the images resampled by interpolation, alleviating the systematic error caused by the sensor difference. The STARFM fusion method uses expert knowledge to extract artificial features in selecting "Spectrally Similar Neighbor Pixels" and computer their weights. Based on the basic concept of STARFM, an automatic feature extraction method using SRCNN as the basic framework is realized. The experimental results show that this method has lower MSE value than the original STARFM, which further improves the quality of spatial-temporal fusion.


Key words: spatial-temporal fusion, improved STARFM, SRCNN, automatic feature extraction