抗噪声的分步式图像超分辨率重构算法

doi:10.13229/j.cnki.jdxbgxb.20221251

摘要/Abstract

摘要：

基于卷积神经网络的图像超分辨率重构方法大多数假设低分辨率图像是从高分辨率图像双三次降采样得到，而现实环境低分辨率图像带有未知噪声，不可避免地导致网络性能较差。针对这一普遍问题，本文提出了一种抗噪声的分步式图像超分辨率重构算法。首先，将信息蒸馏图像降噪网络结合生成对抗网络进行网络训练，以提高降噪网络的图像降噪能力；其次，将降噪网络的中间网络纯净特征图和降噪后的图像与分步式图像超分辨率重构网络结合，配合分步式网络训练，实现网络对真实环境低分辨率图像的有效超分辨率重构。在自建含有高斯噪声的BSD100*与BSD100^#数据集上对本文提出的网络进行了训练和评估。实验结果表明：所提网络与已有先进网络相比，在图像质量评估和视觉对比上均取得较大提升。

关键词: 深度学习, 图像降噪网络, 图像超分辨率重构网络, 生成对抗网络, 分步式网络训练

Abstract:

The image super-resolution reconstruction methods based on convolutional neural network mostly assume that a low resolution （LR） image is obtained by Bicubic downsampling of high resolution （HR） image. However， LR images in real environment contain unknown noise， which inevitably leads to poor network performance. To solve this common problem， A noise-resistant multistep image super resolution network is proposed. First of all， combine information distilling image denoising network and generative adversarial network to train the denoising network， in order to improve image denoising ability of the network； Secondly， the pure feature map in the middle layer of the network and the denoised image are combined with the stepwise image super-resolution reconstruction network， which is combined with the stepwise network training， to reconstruct low-resolution image of the real environment effectively. The proposed network is trained and evaluated on BSD100* and BSD100^# datasets with gaussian noise. Experimental results show that the proposed network achieves good improvement in image quality evaluation and visual comparison compared with existing advanced networks.

Key words: deep learning, image denoising network, image super-resolution reconstruction network, generative adversarial network, stepwise network training

中图分类号:

TP391

郭昕刚,何颖晨,程超. 抗噪声的分步式图像超分辨率重构算法[J]. 吉林大学学报(工学版), 2024, 54(7): 2063-2071.

Xin-gang GUO,Ying-chen HE,Chao CHENG. Noise-resistant multistep image super resolution network[J]. Journal of Jilin University(Engineering and Technology Edition), 2024, 54(7): 2063-2071.

图/表 8

图1

图2

图3

表1

定量评价本文网络与已有的SR网络：2、4倍尺度因子的平均PSNR/SSIM"

算法	Scale	SET14	BSD100	（ $ρ = 15$ ） BSD100^#
Bicubic	×2	30.24/0.867	29.56/0.843	25.26/0.660
DRCN^［24］	×2	32.96/0.910	31.85/0.894	29.13/0.779
LapSRN^［25］	×2	33.02/0.912	31.77/0.891	29.08/0.782
DRRN^［26］	×2	33.14/0.912	31.97/0.894	29.11/0.773
IDN^［27］	×2	33.22/0.913	32.04/0.898	29.17/0.786
LNLN^［28］	×2	33.48/0.914	32.11/0.899	31.24/0.839
RSAN^［29］	×2	33.49/0.914	32.12/0.900	31.32/0.873
NRMSRN	×2	33.53/0.918	32.16/0.906	32.14/0.904
Bicubic	×4	26.00/0.703	25.96/0.669	23.63/0.526
DRCN	×4	28.04/0.766	27.23/0.721	24.33/0.568
LapSRN	×4	28.17/0.769	27.30/0.723	24.36/0.571
DRRN	×4	28.14/0.768	27.32/0.724	24.21/0.564
IDN	×4	28.25/0.771	27.38/0.728	24.40/0.575
LNLN	×4	28.42/0.781	27.45/0.734	26.47/0.637
RSAN	×4	28.44/0.783	27.46/0.736	26.53/0.681
NRMSRN	×4	28.48/0.784	27.51/0.740	27.49/0.738

表1

图4

图5

图6

表2

降噪模块结合GAN网络与未结合GAN网络的NRMSRN在4倍尺度因子的平均PSNR/SSIM"

BSD100^#	有GAN	无GAN
$ρ =$ 5	27.51/0.741	27.47/0.739
$ρ =$ 15	27.51/0.740	27.46/0.735
$ρ = 2$ 5	27.49/0.738	27.43/0.729

表2

参考文献 30

1	刘哲, 刘政, 王恩. 一种基于双段深度残差卷积网的强噪声超分辨率重建算法[J]. 武汉大学学报: 工学版, 2022, 55(3): 300-309.
	Liu Zhe, Liu Zheng, Wang En.A super-resolution reconstruction algorithm of strong noise based on two-segment deep residual convolutional network[J].Journal of Wuhan University (Engineering Edition), 2022, 55(3): 300-309.
2	Dong C, Loy C C, He K, et al. Image super-resolution using deep convolutional networks[J]. IEEE Trans Pattern Anal Mach Intell, 2016, 38(2): 295-307.
3	Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition[DB/OL].[2022-09-10]. .
4	Kim J, Lee J K, Lee K M. Accurate image super-resolution using very deep convolutional networks[C]∥2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, USA,2016:1646-1654.
5	Mao X J, Shen C, Yang Y B. Image restoration using very deep convolutional encoder-decoder networks with symmetric skip connections[DB/OL].[2022-09-12]..
6	Ying T, Jian Y, Liu X. Image super-resolution via deep recursive residual network[C]∥2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, USA, 2017: No.298.
7	Lim B, Son S, Kim H, et al. Enhanced deep residual networks for single image super-resolution[C]∥2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Honolulu, USA, 2017: No.151.
8	Zhang Y, Li K, Li K, et al. Image super-resolution using very deep residual channel attention networks[C]∥European Conference on Computer Vision, Munich,Germany,2018: 294-310.
9	Dai T, Cai J, Zhang Y, et al. Second-order attention network for single image super-resolution[C]∥2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Long Beach, USA, 2019: No.01132.
10	Tong T, Li G, Liu X, et al. Image super-resolution using dense skip connections[C]∥2017 IEEE International Conference on Computer Vision (ICCV),Venice, Italy, 2017: No.514.
11	Tai Y, Yang J, Liu X, et al. MemNet: a persistent memory network for image restoration[C]∥2017 IEEE International Conference on Computer Vision (ICCV), Venice, Italy, 2017: No.486.
12	Hui Z, Li J, Gao X, et al. Progressive perception-oriented network for single image super-resolution[DB/OL]. [2022-09-12]..
13	Zhuo S, Jin Z, Zou W, et al. RIDNet: recursive information distillation network for color image denoising[C]∥2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW), Seoul, Korea (South), 2019: No.00483.
14	Xu Q, Zhang C, Li Z. Denoising convolutional neural network[C]∥2015 IEEE International Conference on Information and Automation, Lijiang, China, 2015: No.7279466.
15	Goodfellow I, Pouget-Abadie J, Mirza M, et al. Generative adversarial nets[C]∥Neural Information Processing Systems. Cambridge: MIT Press, 2014: 2672-2680.
16	Ledig C, Theis L, Huszar F, et al. Photo-realistic single image super-resolution using a generative adversarial network[C]∥2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, USA, 2017: No.19.
17	Wang X, Yu K, Wu S, et al. ESRGAN: enhanced super-resolution generative adversarial networks[C]∥Computer Vision-ECCV 2018 Workshops.Berlin:Springer, 2018: 63-79.
18	Jolicoeur-Martineau A. The relativistic dis-criminator: a key element missing from standard GAN[DB/OL].[2022-09-12]..
19	Oyedotun O K, Ismaeil K A, Aouada D. Training very deep neural networks: rethinking the role of skip connections[J]. Neurocomputing, 2021,441(Jun.21):105-117.
20	Woo S, Park J, Lee J Y, et al. CBAM: convolutional block attention module[M]. Berlin: Springer, 2018.
21	Shi W, Caballero J, Huszár F, et al. Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network[DB/OL].[2022-09-13]..
22	Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition[DB/OL].[2022-09-13]..
23	Dong C, Loy C C, He K, et al. Image super-resolution using deep convolutional networks[J]. TPAMI, 2015, 38(2): 295-307.
24	Kim J, Lee J K, Lee K M. Deeply-recursive convolutional network for image super-resolution[J]. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, USA, 2016: No.181.
25	Lai W S, Huang J B, Ahuja N, et al. Deep laplacian pyramid networks for fast and accurate super-resolution[C]∥2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR),Honolulu, USA,2017: 5835-5843.
26	Tai Y, Yang J, Liu X. Image super-resolution via deep recursive residual network[C]∥IEEE Conference on Computer Vision and Pattern Recognition,Honolulu, USA, 2017: 3147-3155.
27	Zheng H, Wang X, Gao X. Fast and accurate single image super-resolution via information distillation network[C]∥2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Salt Lake City, USA, 2018: No.00082.
28	Wang R, Lei T, Zhou W, et al. Lightweight non-local network for image super-resolution[C]∥International Conference on Acoustics, Speech, and Signal Processing, New Orleans, USA, 2021: 1625-1629.
29	Wei S, Sun X, Zhao H R,et al. RSAN: residual subtraction and attention network for single image super-resolution[C]∥IEEE International Conference on Pattern Recognition, Milan, Italy, 2021.
30	Wang Z, Bovik A C, Sheikh H R,et al. Image quality assessment: from error visibility to structural similarity[J]. TIP, 2006, 13(4): 600-612.

相关文章 15

[1]	乔百友,武彤,杨璐,蒋有文. 一种基于BiGRU和胶囊网络的文本情感分析方法[J]. 吉林大学学报(工学版), 2024, 54(7): 2026-2037.
[2]	孙铭会,薛浩,金玉波,曲卫东,秦贵和. 联合时空注意力的视频显著性预测[J]. 吉林大学学报(工学版), 2024, 54(6): 1767-1776.
[3]	张丽平,刘斌毓,李松,郝忠孝. 基于稀疏多头自注意力的轨迹kNN查询方法[J]. 吉林大学学报(工学版), 2024, 54(6): 1756-1766.
[4]	陆玉凯,袁帅科,熊树生,朱绍鹏,张宁. 汽车漆面缺陷高精度检测系统[J]. 吉林大学学报(工学版), 2024, 54(5): 1205-1213.
[5]	李雄飞,宋紫萱,朱芮,张小利. 基于多尺度融合的遥感图像变化检测模型[J]. 吉林大学学报(工学版), 2024, 54(2): 516-523.
[6]	杨国俊,齐亚辉,石秀名. 基于数字图像技术的桥梁裂缝检测综述[J]. 吉林大学学报(工学版), 2024, 54(2): 313-332.
[7]	陈岳林,高铸成,蔡晓东. 基于BERT与密集复合网络的长文本语义匹配模型[J]. 吉林大学学报(工学版), 2024, 54(1): 232-239.
[8]	霍光,林大为,刘元宁,朱晓冬,袁梦,盖迪. 基于多尺度特征和注意力机制的轻量级虹膜分割模型[J]. 吉林大学学报(工学版), 2023, 53(9): 2591-2600.
[9]	金小俊,孙艳霞,于佳琳,陈勇. 基于深度学习与图像处理的蔬菜苗期杂草识别方法[J]. 吉林大学学报(工学版), 2023, 53(8): 2421-2429.
[10]	耿庆田,刘植,李清亮,于繁华,李晓宁. 基于一种深度学习模型的土壤湿度预测[J]. 吉林大学学报(工学版), 2023, 53(8): 2430-2436.
[11]	巫威眺,曾坤,周伟,李鹏,靳文舟. 基于多源数据和响应面优化的公交客流预测深度学习方法[J]. 吉林大学学报(工学版), 2023, 53(7): 2001-2015.
[12]	张振海,季坤,党建武. 基于桥梁裂缝识别模型的桥梁裂缝病害识别方法[J]. 吉林大学学报(工学版), 2023, 53(5): 1418-1426.
[13]	吴飞,农皓业,马晨浩. 基于粒子群优化算法⁃长短时记忆模型的刀具磨损预测方法[J]. 吉林大学学报(工学版), 2023, 53(4): 989-997.
[14]	吕卫,韩镓泽,褚晶辉,井佩光. 基于多模态自注意力网络的视频记忆度预测[J]. 吉林大学学报(工学版), 2023, 53(4): 1211-1219.
[15]	何科,丁海涛,赖宣淇,许男,郭孔辉. 基于Transformer的轮式里程计误差预测模型[J]. 吉林大学学报(工学版), 2023, 53(3): 653-662.

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed