结合坐标注意力与生成式对抗网络的图像超分辨率重建

计算机工程与科学 ›› 2024, Vol. 46 ›› Issue (01): 122-131.

结合坐标注意力与生成式对抗网络的图像超分辨率重建

彭晏飞，孟欣，李泳欣，刘蓝兮

（辽宁工程技术大学电子与信息工程学院，辽宁葫芦岛 125100）

收稿日期:2023-03-06 修回日期:2023-05-16 接受日期:2024-01-25 出版日期:2024-01-25 发布日期:2024-01-15
基金资助:
国家自然科学基金（61772249）；辽宁省高等学校基本科研项目（LJKZ0358,LJKQZ2021152）

Combining coordinate attention and generative adversarial network for image super-resolution reconstruction

PENG Yan-fei,MENG Xin,LI Yong-xin,LIU Lan-xi

(School of Electronics and Information Engineering,Liaoning Technical University,Huludao 125100,China)

Received:2023-03-06 Revised:2023-05-16 Accepted:2024-01-25 Online:2024-01-25 Published:2024-01-15

摘要/Abstract

摘要： 针对现有生成式对抗网络GAN的图像超分辨率重建模型中存在着特征信息利用不充分、VGG式判别器对局部细节的判断能力较弱以及训练不稳定的问题，提出了一种结合坐标注意力与生成式对抗网络的图像超分辨率重建模型。首先，以嵌有坐标注意力的残差块构建生成器，沿通道和空间2个维度聚合特征，更充分地提取特征。然后，调整Dropout加入网络的方式使其作用于生成器中，提高模型的泛化能力。接着，以U-Net结构构造判别器，输出详细的逐像素反馈，以获取真假图像间的局部差异。最后，在判别器中引入谱归一化正则化，稳定GAN的训练。实验结果表明，当放大因子为4时，在基准测试集Set5和Set14上取得的峰值信噪比平均提高了1.75 dB，结构相似性平均提高了0.038，能够重建出更加清晰且真实的图像，重建图像具有良好的视觉效果。

关键词: 超分辨率重建, 生成式对抗网络, 坐标注意力, U-Net式判别器

Abstract: An image super-resolution reconstruction model combining coordinate attention and generative adversarial networks is proposed to address the problems of inadequate utilization of feature information, weak judgment of local details by VGG discriminators, and unstable training in the existing image super-resolution reconstruction model of generative adversarial networks. Firstly, a generator is constructed with residual blocks embedded with coordinate attention to aggregate features along both channel and spatial dimensions to extract features more adequately. The Dropout is also adjusted to join the network in such a way that it acts in the generator to improve the generalization ability of the model. Secondly, the discriminator is constructed with U-Net structure to output detailed pixel-by-pixel feedback to obtain the local difference between the true and false images. Finally, spectral normalization regularization is introduced into the discriminator to stabilize the training of GAN. The experimental results show that when the amplification factor is 4, the peak signal-to-noise ratio obtained on the benchmark test sets Set5 and Set14 is increased by 1.75 dB on average, and the structural similarity is increased by 0.038 on average, which can reconstruct clearer and more realistic images with good visual effects.

Key words: super-resolution reconstruction, generative adversarial network, coordinate attention, U-Net discriminator

彭晏飞, 孟欣, 李泳欣, 刘蓝兮. 结合坐标注意力与生成式对抗网络的图像超分辨率重建[J]. 计算机工程与科学, 2024, 46(01): 122-131.

PENG Yan-fei, MENG Xin, LI Yong-xin, LIU Lan-xi. Combining coordinate attention and generative adversarial network for image super-resolution reconstruction[J]. Computer Engineering & Science, 2024, 46(01): 122-131.

参考文献［26］

［1］	Yu Y L, Li X Z, Liu F X. E-DBPN:Enhanced deep back- projection networks for remote sensing scene image superresolution［J］.IEEE Transactions on Geoscience and Remote Sensing,2020,58(8):5503-5515.
［2］	Chen L,Pan J,Li Q.Robust face image super-resolution via joint learning of subdivided contextual model［J］.IEEE Transactions on Image Processing,2019,28(12):5897-5909.
［3］	You C Y,Li G,Zhang Y,et al.CT super-resolution GAN constrained by the identical,residual,and cycle learning ensemble ［J］. IEEE Transactions on Medical Imaging,2020,39(1):188-203.
［4］	Singh A,Singh J.Content adaptive single image interpolation based super resolution of compressed images［J］.International Journal of Electrical and Computer Engineering,2020,10(3):3014-3021.
［5］	Ma X, Zhang J, Li T, et al. Super-resolution geomagnetic reference map reconstruction based on dictionary learning and sparse representation［J］.IEEE Access,2020,8:84316-84325.
［6］	Kong X T,Zhao H Y,Qiao Y,et al.ClassSR:A general framework to accelerate super-resolution networks by data characteristic［C］∥Proc of 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition,2021:12011-12020.
［7］	Dong C, Loy C C,He K M,et al.Image super-resolution using deep convolutional networks［J］.IEEE Transactions on Pattern Analysis and Machine Intelligence,2016,38(2):295-307.
［8］	Kim J, Lee J K,Lee K M.Accurate image super-resolution using very deep convolutional networks［C］∥Proc of 2016 IEEE Conference on Computer Vision and Pattern Recognition,2016:1646-1654.
［9］	He K N,Zhang X Y,Ren S Q,et al.Deep residual learning for image recognition［C］∥Proc of 2016 IEEE Conference on Computer Vision and Pattern Recognition,2016:770-778.
［10］	Kim J,Lee J K,Lee K M.Deeply-recursive convolutional network for image super-resolution［C］∥Proc of 2016 IEEE Conference on Computer Vision and Pattern Recognition,2016:1637-1645.
［11］	Shi W,Caballero J,Huszar F,et al.Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network［C］∥Proc of 2016 IEEE Confe- rence on Computer Vision and Pattern Recognition,2016:1871-1883.
［12］	Zhang Y L,Li K P,Li K,et al.Image super-resolution using very deep residual channel attention networks［C］∥Proc of the 15th European Conference on Computer Vision,2018:294-310.
［13］	Goodfellow I,Pouget-Abadie J,Mirza M,et al.Generative adversarial nets［C］∥Proc of the 27th International Confe- rence on Neural Information Processing Systems,2014:2672-2680.
［14］	Ledig C,Theis L,Huszár F,et al.Photo-realistic single image super-resolution using a generative adversarial network［C］∥Proc of 2017 IEEE Conference on Computer Vision and Pattern Recognition,2017:105-114.
［15］	Wang X T, Yu K,Wu S X,et al. ESRGAN:Enhanced super-resolution generative adversarial networks［C］∥Proc of the 15th European Conference on Computer Vision Workshop,2018:63-79.
［16］	Li W,Zhou K,Qi L,et al.Best-buddy GANs for highly detailed image super-resolution［J］.arXiv:2103.15295,2021.
［17］	Yan Y T,Liu C C,Chen C Y,et al.Fine-grained attention and feature-sharing generative adversarial networks for single image super-resolution［J］.IEEE Transactions on Multimedia,2022,24:1473-1487.
［18］	Wang X,Xie L,Dong C,et al.Real-ESRGAN:Training real-world blind super-resolution with pure synthetic data［C］∥Proc of 2021 IEEE/CVF International Conference on Computer Vision Workshops,2021:1905-1914.
［19］	Hou Q, Zhou D,Feng J.Coordinate attention for efficient mobile network design［J］.arXiv:2103.02907,2021.
［20］	Kong X T,Liu X N,Gu J J,et al.Reflash dropout in image super-resolution［C］∥Proc of 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition,2022:5992-6002.
［21］	Hinton G,Srivastava N,Krizhevsky A,et al.Improving neural networks by preventing co-adaptation of feature detectors［J］.arXiv:1207.0580,2012.
［22］	Schnfeld E,Schiele B,Khoreva A.A U-Net based discriminator for generative adversarial networks［C］∥Proc of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition,2020:8204-8213.
［23］	Arjovsky M,Chintala S,Bottou L,et al.Wasserstein generative adversarial networks［C］∥Proc of the 34th International Conference on Machine Learning,2017:214-223.
［24］	Miyato T,Kataoka T,Koyama M,et al.Spectral normalization for generative adversarial networks［J］.arXiv:1802.05957,2018.
［25］	Barron J T.A general and adaptive robust loss function［C］∥Proc of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition,2019:4326-4334.
［26］	Timofte R,Agustsson E, van Gool L,et al.NTIRE 2017 challenge on single image super-resolution:Methods and results［C］∥Proc of 2017 IEEE Conference on Computer Vision and Pattern Recognition,2017:1110-1121.