• 中国计算机学会会刊
  • 中国科技核心期刊
  • 中文核心期刊

计算机工程与科学 ›› 2021, Vol. 43 ›› Issue (12): 2216-2222.

• 图形与图像 • 上一篇    下一篇

一种结合自注意和多尺度生成对抗网络的图像去雨方法

李然,周子淏,张月芳,罗东升,邓红霞   

  1. (太原理工大学信息与计算机学院,山西 晋中 030600)

  • 收稿日期:2020-09-03 修回日期:2020-12-18 接受日期:2021-12-25 出版日期:2021-12-25 发布日期:2021-12-31
  • 基金资助:
    山西省重点研发计划(201803D31038);晋中市科技重点研发项目(Y192006);山西省自然科学基金(201801D121135)

An image raindrop removal method based on self-attention and multi-scale generative adversarial network

LI Ran,ZHOU Zi-hao,ZHANG Yue-fang,LUO Dong-sheng,DENG Hong-xia#br# #br# #br#   

  1. (College of Information and Computer,Taiyuan University of Technology,Jinzhong 030600,China)
  • Received:2020-09-03 Revised:2020-12-18 Accepted:2021-12-25 Online:2021-12-25 Published:2021-12-31

摘要: 为去除雨天拍摄照片上的雨滴,针对被雨滴所覆盖区域未知,雨滴区域中大多数背景信息已经丢失,以及需要提升图像清晰度和对全局信息关注度的问题,在生成对抗网络中生成网络的自动编码器结构中添加自注意层,并在判别网络中引入多尺度判别器。通过注意力分布图的引导,自注意层的优化和多尺度判别器的评估,生成网络在关注雨滴区域的前提下进一步关注全局信息,多尺度判别器可由粗到细更好地判别雨滴图像与清晰图像之间的差距。实验完成了所提方法与其他方法的对比,以及自对比,并用峰值信噪比和结构相似性进行评估,结果表明了所提方法的有效性,其质量和指标数值均高于其他方法。

关键词: 去雨, 生成对抗网络, 自我注意, 多尺度

Abstract: In order to remove raindrops from images taken on rainy days, aiming at the issues that the area covered by raindrops is unknown, most of the background information in the raindrop area has been lost, and the image clarity and global information attention are needed to improve, a self-attention layer is added to the self-encoding structure, and a multi-scale discriminator is introduced into the discriminant network. Guided by the attention distribution map, the optimization of the self-attention layer and the evaluation of the multi-scale discriminator, the generating network considers the global information more under the premise of paying attention to the raindrop area. The multi-scale discriminator can better distinguish the gap between the raindrop image and the clear image from coarsely to finely. The experiment completed the comparison between the proposed method and other methods, the self- comparison, and the evaluation with the peak signal-to-noise ratio and structural similarity, which proves that the proposed method is effective and its quality and index values are higher than other methods.


Key words: raindrop removal, generative adversarial network, self-attention, multi-scale