抗噪声的分步式图像超分辨率重构算法

doi:10.13229/j.cnki.jdxbgxb.20221251

Abstract

Abstract:

The image super-resolution reconstruction methods based on convolutional neural network mostly assume that a low resolution （LR） image is obtained by Bicubic downsampling of high resolution （HR） image. However， LR images in real environment contain unknown noise， which inevitably leads to poor network performance. To solve this common problem， A noise-resistant multistep image super resolution network is proposed. First of all， combine information distilling image denoising network and generative adversarial network to train the denoising network， in order to improve image denoising ability of the network； Secondly， the pure feature map in the middle layer of the network and the denoised image are combined with the stepwise image super-resolution reconstruction network， which is combined with the stepwise network training， to reconstruct low-resolution image of the real environment effectively. The proposed network is trained and evaluated on BSD100* and BSD100^# datasets with gaussian noise. Experimental results show that the proposed network achieves good improvement in image quality evaluation and visual comparison compared with existing advanced networks.

Key words: deep learning, image denoising network, image super-resolution reconstruction network, generative adversarial network, stepwise network training

CLC Number:

TP391

Xin-gang GUO,Ying-chen HE,Chao CHENG. Noise-resistant multistep image super resolution network[J].Journal of Jilin University(Engineering and Technology Edition), 2024, 54(7): 2063-2071.

Figures/Tables 8

Fig. 1

Fig.2

Fig.3

Table 1

Mean PSNR/SSIM of 2× and 4× scale factorswere quantitatively evaluated for proposednetwork and existing SR network"

算法	Scale	SET14	BSD100	（ $ρ = 15$ ） BSD100^#
Bicubic	×2	30.24/0.867	29.56/0.843	25.26/0.660
DRCN^［24］	×2	32.96/0.910	31.85/0.894	29.13/0.779
LapSRN^［25］	×2	33.02/0.912	31.77/0.891	29.08/0.782
DRRN^［26］	×2	33.14/0.912	31.97/0.894	29.11/0.773
IDN^［27］	×2	33.22/0.913	32.04/0.898	29.17/0.786
LNLN^［28］	×2	33.48/0.914	32.11/0.899	31.24/0.839
RSAN^［29］	×2	33.49/0.914	32.12/0.900	31.32/0.873
NRMSRN	×2	33.53/0.918	32.16/0.906	32.14/0.904
Bicubic	×4	26.00/0.703	25.96/0.669	23.63/0.526
DRCN	×4	28.04/0.766	27.23/0.721	24.33/0.568
LapSRN	×4	28.17/0.769	27.30/0.723	24.36/0.571
DRRN	×4	28.14/0.768	27.32/0.724	24.21/0.564
IDN	×4	28.25/0.771	27.38/0.728	24.40/0.575
LNLN	×4	28.42/0.781	27.45/0.734	26.47/0.637
RSAN	×4	28.44/0.783	27.46/0.736	26.53/0.681
NRMSRN	×4	28.48/0.784	27.51/0.740	27.49/0.738

Table 1

Fig.4

Fig.5

Fig. 6

Table 2

Mean PSNR/SSIM of 4× scale factors of NRMSRN combined with GAN network and without GAN network in denoising module"

BSD100^#	有GAN	无GAN
$ρ =$ 5	27.51/0.741	27.47/0.739
$ρ =$ 15	27.51/0.740	27.46/0.735
$ρ = 2$ 5	27.49/0.738	27.43/0.729

Table 2

References 30

1	刘哲, 刘政, 王恩. 一种基于双段深度残差卷积网的强噪声超分辨率重建算法[J]. 武汉大学学报: 工学版, 2022, 55(3): 300-309.
	Liu Zhe, Liu Zheng, Wang En.A super-resolution reconstruction algorithm of strong noise based on two-segment deep residual convolutional network[J].Journal of Wuhan University (Engineering Edition), 2022, 55(3): 300-309.
2	Dong C, Loy C C, He K, et al. Image super-resolution using deep convolutional networks[J]. IEEE Trans Pattern Anal Mach Intell, 2016, 38(2): 295-307.
3	Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition[DB/OL].[2022-09-10]. .
4	Kim J, Lee J K, Lee K M. Accurate image super-resolution using very deep convolutional networks[C]∥2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, USA,2016:1646-1654.
5	Mao X J, Shen C, Yang Y B. Image restoration using very deep convolutional encoder-decoder networks with symmetric skip connections[DB/OL].[2022-09-12]..
6	Ying T, Jian Y, Liu X. Image super-resolution via deep recursive residual network[C]∥2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, USA, 2017: No.298.
7	Lim B, Son S, Kim H, et al. Enhanced deep residual networks for single image super-resolution[C]∥2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Honolulu, USA, 2017: No.151.
8	Zhang Y, Li K, Li K, et al. Image super-resolution using very deep residual channel attention networks[C]∥European Conference on Computer Vision, Munich,Germany,2018: 294-310.
9	Dai T, Cai J, Zhang Y, et al. Second-order attention network for single image super-resolution[C]∥2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Long Beach, USA, 2019: No.01132.
10	Tong T, Li G, Liu X, et al. Image super-resolution using dense skip connections[C]∥2017 IEEE International Conference on Computer Vision (ICCV),Venice, Italy, 2017: No.514.
11	Tai Y, Yang J, Liu X, et al. MemNet: a persistent memory network for image restoration[C]∥2017 IEEE International Conference on Computer Vision (ICCV), Venice, Italy, 2017: No.486.
12	Hui Z, Li J, Gao X, et al. Progressive perception-oriented network for single image super-resolution[DB/OL]. [2022-09-12]..
13	Zhuo S, Jin Z, Zou W, et al. RIDNet: recursive information distillation network for color image denoising[C]∥2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW), Seoul, Korea (South), 2019: No.00483.
14	Xu Q, Zhang C, Li Z. Denoising convolutional neural network[C]∥2015 IEEE International Conference on Information and Automation, Lijiang, China, 2015: No.7279466.
15	Goodfellow I, Pouget-Abadie J, Mirza M, et al. Generative adversarial nets[C]∥Neural Information Processing Systems. Cambridge: MIT Press, 2014: 2672-2680.
16	Ledig C, Theis L, Huszar F, et al. Photo-realistic single image super-resolution using a generative adversarial network[C]∥2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, USA, 2017: No.19.
17	Wang X, Yu K, Wu S, et al. ESRGAN: enhanced super-resolution generative adversarial networks[C]∥Computer Vision-ECCV 2018 Workshops.Berlin:Springer, 2018: 63-79.
18	Jolicoeur-Martineau A. The relativistic dis-criminator: a key element missing from standard GAN[DB/OL].[2022-09-12]..
19	Oyedotun O K, Ismaeil K A, Aouada D. Training very deep neural networks: rethinking the role of skip connections[J]. Neurocomputing, 2021,441(Jun.21):105-117.
20	Woo S, Park J, Lee J Y, et al. CBAM: convolutional block attention module[M]. Berlin: Springer, 2018.
21	Shi W, Caballero J, Huszár F, et al. Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network[DB/OL].[2022-09-13]..
22	Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition[DB/OL].[2022-09-13]..
23	Dong C, Loy C C, He K, et al. Image super-resolution using deep convolutional networks[J]. TPAMI, 2015, 38(2): 295-307.
24	Kim J, Lee J K, Lee K M. Deeply-recursive convolutional network for image super-resolution[J]. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, USA, 2016: No.181.
25	Lai W S, Huang J B, Ahuja N, et al. Deep laplacian pyramid networks for fast and accurate super-resolution[C]∥2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR),Honolulu, USA,2017: 5835-5843.
26	Tai Y, Yang J, Liu X. Image super-resolution via deep recursive residual network[C]∥IEEE Conference on Computer Vision and Pattern Recognition,Honolulu, USA, 2017: 3147-3155.
27	Zheng H, Wang X, Gao X. Fast and accurate single image super-resolution via information distillation network[C]∥2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Salt Lake City, USA, 2018: No.00082.
28	Wang R, Lei T, Zhou W, et al. Lightweight non-local network for image super-resolution[C]∥International Conference on Acoustics, Speech, and Signal Processing, New Orleans, USA, 2021: 1625-1629.
29	Wei S, Sun X, Zhao H R,et al. RSAN: residual subtraction and attention network for single image super-resolution[C]∥IEEE International Conference on Pattern Recognition, Milan, Italy, 2021.
30	Wang Z, Bovik A C, Sheikh H R,et al. Image quality assessment: from error visibility to structural similarity[J]. TIP, 2006, 13(4): 600-612.

Related Articles 15

[1]	Bai-you QIAO,Tong WU,Lu YANG,You-wen JIANG. A text sentiment analysis method based on BiGRU and capsule network [J]. Journal of Jilin University(Engineering and Technology Edition), 2024, 54(7): 2026-2037.
[2]	Ming-hui SUN,Hao XUE,Yu-bo JIN,Wei-dong QU,Gui-he QIN. Video saliency prediction with collective spatio-temporal attention [J]. Journal of Jilin University(Engineering and Technology Edition), 2024, 54(6): 1767-1776.
[3]	Li-ping ZHANG,Bin-yu LIU,Song LI,Zhong-xiao HAO. Trajectory k nearest neighbor query method based on sparse multi-head attention [J]. Journal of Jilin University(Engineering and Technology Edition), 2024, 54(6): 1756-1766.
[4]	Yu-kai LU,Shuai-ke YUAN,Shu-sheng XIONG,Shao-peng ZHU,Ning ZHANG. High precision detection system for automotive paint defects [J]. Journal of Jilin University(Engineering and Technology Edition), 2024, 54(5): 1205-1213.
[5]	Xiong-fei LI,Zi-xuan SONG,Rui ZHU,Xiao-li ZHANG. Remote sensing change detection model based on multi⁃scale fusion [J]. Journal of Jilin University(Engineering and Technology Edition), 2024, 54(2): 516-523.
[6]	Guo-jun YANG,Ya-hui QI,Xiu-ming SHI. Review of bridge crack detection based on digital image technology [J]. Journal of Jilin University(Engineering and Technology Edition), 2024, 54(2): 313-332.
[7]	Yue-lin CHEN,Zhu-cheng GAO,Xiao-dong CAI. Long text semantic matching model based on BERT and dense composite network [J]. Journal of Jilin University(Engineering and Technology Edition), 2024, 54(1): 232-239.
[8]	Guang HUO,Da-wei LIN,Yuan-ning LIU,Xiao-dong ZHU,Meng YUAN,Di GAI. Lightweight iris segmentation model based on multiscale feature and attention mechanism [J]. Journal of Jilin University(Engineering and Technology Edition), 2023, 53(9): 2591-2600.
[9]	Xiao-jun JIN,Yan-xia SUN,Jia-lin YU,Yong CHEN. Weed recognition in vegetable at seedling stage based on deep learning and image processing [J]. Journal of Jilin University(Engineering and Technology Edition), 2023, 53(8): 2421-2429.
[10]	Qing-tian GENG,Zhi LIU,Qing-liang LI,Fan-hua YU,Xiao-ning LI. Prediction of soil moisture based on a deep learning model [J]. Journal of Jilin University(Engineering and Technology Edition), 2023, 53(8): 2430-2436.
[11]	Wei-tiao WU,Kun ZENG,Wei ZHOU,Peng LI,Wen-zhou JIN. Deep learning method for bus passenger flow prediction based on multi-source data and surrogate-based optimization [J]. Journal of Jilin University(Engineering and Technology Edition), 2023, 53(7): 2001-2015.
[12]	Zhen-hai ZHANG,Kun JI,Jian-wu DANG. Crack identification method for bridge based on BCEM model [J]. Journal of Jilin University(Engineering and Technology Edition), 2023, 53(5): 1418-1426.
[13]	Fei WU,Hao-ye NONG,Chen-hao MA. Tool wear prediction method based on particle swarm optimizationlong and short time memory model [J]. Journal of Jilin University(Engineering and Technology Edition), 2023, 53(4): 989-997.
[14]	Wei LYU,Jia-ze HAN,Jing-hui CHU,Pei-guang JING. Multi⁃modal self⁃attention network for video memorability prediction [J]. Journal of Jilin University(Engineering and Technology Edition), 2023, 53(4): 1211-1219.
[15]	Wen-li JI,Zhong TIAN,Jing CHAI,Ding-ding ZHANG,Bin WANG. Prediction of water⁃flowing height in fractured zone based on distributed optical fiber and multi⁃attribute fusion [J]. Journal of Jilin University(Engineering and Technology Edition), 2023, 53(4): 1200-1210.

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed

Comments

Recommended 0

No Suggested Reading articles found!

Noise-resistant multistep image super resolution network

RICH HTML

PDF (PC)