基于残差连接卷积神经网络的图像超分辨率重构

doi:10.13229/j.cnki.jdxbgxb20180366

Abstract

Abstract:

Super-resolution reconstruction algorithms based on convolutional neural network are confront the problems of small receptive field, disappearing of gradient information and slow convergence of the network. In order to solve these problems, an image super-resolution reconstruction algorithm based on residual connection convolutional neural network is proposed. Processing images in high resolution space results in the increase in network complexity. So, images are processed in low-resolution space to reduce complexity. The convolutional neural network is improved by introducing local and global residuals. Local residual promotes the flow of information and avoids the problem of gradient disappearing during network training. Due to the local residual connection, image information can be transferred to deeper layers. The global residual makes the network learn the residual information in images only, reducing network redundancy and accelerating network convergence. The receptive field is expanded by increasing the network depth, which makes the network learn more reconstruction information. The PSNR and SSIM of the proposed algorithm are comparable to other algorithms.

Key words: imformation processing technology, image super-resolution, convolution neural network, subpixel upsampling, residual connection

CLC Number:

TP751.1

Ji-chang GUO,Jie WU,Chun-le GUO,Ming-hui ZHU. Image super-resolution reconstruction based on residual connection convolutional neural network[J].Journal of Jilin University(Engineering and Technology Edition), 2019, 49(5): 1726-1734.

Figures/Tables 12

Fig.1

Fig.2

Fig.3

Fig.4

Table 1

Fig.5

Table 2

Fig. 6

Fig. 7

Fig.8

Fig.9

Table 3

References 19

1	Tsai R Y , Huang T S . Multiframe image restoration and registration[J]. Advance Computer Visual and Image Processing, 1984, 1(2): 317-339.
2	Peleg S , Keren D , Schweitzer L . Improving image resolution using subpixel motion[J]. Pattern Recognition Letters, 1987, 5(3): 223-226.
3	Schultz R R , Stevenson R L . Extraction of high-resolution frames from video sequences[J]. IEEE Transactions on Image Processing, 1996, 5(6): 996-1011.
4	Freeman W T , Jones T R , Pasztor E C . Example-based super-resolution[J]. IEEE Computer Graphics & Applications, 2002, 22(2): 56-65.
5	薛翠红，于明，杨宇皓, 等 . 基于学习的马尔科夫超分辨率复原[J]. 吉林大学学报: 工学版， 2013, 43(增刊1): 406-409.
	Xue Cui-hong , Yu Ming , Yang Yu-hao , et al . MRF reconstruction based on Markov network[J]. Journal of Jilin University (Engineering and Technology Edition), 2013, 43(Sup.1): 406-409.
6	徐岩, 孙美双 . 基于卷积神经网络的水下图像增强方法[J]. 吉林大学学报：工学版, 2018, 48(6): 1895-1903.
	Xu Yan , Sun Mei-shuang . Enhancing underwater image based on convolutional neural networks[J]. Journal of Jilin University (Engineering and Technology Edition), 2018, 48(6): 1895-1903.
7	马淼, 李贻斌 . 基于多级图像序列和卷积神经网络的人体行为识别[J]. 吉林大学学报：工学版, 2017, 47(4): 1244-1252.
	Ma Miao , Li Yi-bin . Multi-level image sequences and convolutional neural networks based human action recognition method[J]. Journal of Jilin University (Engineering and Technology Edition)， 2017, 47(4): 1244-1252.
8	Dong C , Chen C L , He K , et al . Learning a deep convolutional network for image super-resolution[C]∥European Conference on Computer Vision. Cham: Springer International Publishing, 2014: 184-199.
9	Dong C , Loy C C , Tang X . Accelerating the super-resolution convolutional neural network[C]∥European Conference on Computer Vision. Amsterdam: Springer International Publishing, 2016: 391-407.
10	Shi W , Caballero J , Huszár F , et al . Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network[C]∥IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas: IEEE Computer Society, 2016: 1874-1883.
11	Kim J , Lee J K , Lee K M . Accurate image super-resolution using very deep convolutional networks[C]∥IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas: IEEE Computer Society, 2016: 1646-1654.
12	Kim J , Kwon Lee J , Mu Lee K . Deeply-recursive convolutional network for image super-resolution[C]∥IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas: IEEE Computer Society, 2016: 1637-1645.
13	Justin J , Alexandre A , Li F F . Perceptual losses for real-time style transfer and super-resolution[C]∥European Conference on Computer Vision. Amsterdam: Springer International Publishing, 2016: 694-711.
14	Ledig C , Wang Z , Shi W , et al . Photo-realistic single image super-resolution using a generative adversarial network[C]∥IEEE Conference on Computer Vision and Pattern Recognition. Hawaii: IEEE Computer Society, 2017: 105-114.
15	Cui Z , Chang H , Shan S , et al . Deep network cascade for image super-resolution[C]∥European Conference on Computer Vision. Cham: Springer International Publishing, 2014: 49-64.
16	Mao X J , Shen C , Yang Y B . Image restoration using very deep convolutional encoder-decoder networks with symmetric skip connections[J]. Advances in Neural Information Processing Systems, 2016(2): 2810-2818.
17	Wang Y F , Wang L J , Wang H Y , et al . End-to-end image super-resolution via deep and shallow convolutional networks[J]. IEEE Access, 2016(7): 959-970.
18	He K , Zhang X , Ren S , et al . Deep residual learning for image recognition[C]∥IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas: IEEE Computer Society, 2016: 770-778.
19	Nah S , Kim T H , Lee K M . Deep multi-scale convolutional neural network for dynamic scene deblurring[C]∥IEEE Conference on Computer Vision and Pattern Recognition. Hawaii: IEEE Computer Society, 2017: 257-265.

Related Articles 8

[1]	WANG Ke-yan, LI Yun-song, SONG Juan, LIAO Hui-lin, WU Xian-yun. Spatial-spectral lossless compression of hyperspectral images using local edge based prediction [J]. 吉林大学学报(工学版), 2017, 47(2): 677-685.
[2]	ZHAO Bo, QIN Gui-he. High robustness image watermarking algorithm [J]. 吉林大学学报(工学版), 2017, 47(1): 249-254.
[3]	WANG Jing-meng, ZHANG Ai-wu, ZHAO Ning-ning, MENG Xian-gang. Influence of tilting angle on tilting sampling aliasing and relationship between aliasing and resolution [J]. 吉林大学学报(工学版), 2015, 45(3): 953-960.
[4]	XIA Ying-jie, LI Jin-ping, CHEN Rui. Automatic synthetic registration method based on substation multi-modal images [J]. 吉林大学学报(工学版), 2013, 43(增刊1): 47-50.
[5]	LI Zhi-min, HOU Yong-hong, WANG Bao-liang, HOU Chun-ping, ZHOU Yuan. Parallax adjustment method for stereoscopic display on mobile phones based on optical-plate [J]. 吉林大学学报(工学版), 2013, 43(增刊1): 304-308.
[6]	WANG Yu-lei, ZHAO Chun-hui, QI Bin. Hyperspectral anomaly detection algorithm based on spectral similarity scale [J]. 吉林大学学报(工学版), 2013, 43(增刊1): 148-153.
[7]	FANG Ying-ying, TENG Qi-zhi, HE Xiao-hai, YANG Xiao-min, LI Zheng-ji. Analysis for three-dimensional reconstruction technique of sandstone [J]. 吉林大学学报(工学版), 2013, 43(增刊1): 11-15.
[8]	QI Bin, ZHAO Chun-hui, WANG Yu-lei. Hyperspectral imagery classification based on SVM and RVM [J]. 吉林大学学报(工学版), 2013, 43(增刊1): 143-147.

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed

Comments

Recommended 0

No Suggested Reading articles found!

数据库	尺度因子	双三次插值PSNR/SSIM	A+ PSNR/SSIM	SRCNN PSNR/SSIM	FSRCNN PSNR/SSIM	本文方法PSNR/SSIM
Set5	×2	33.66/0.992	36.54/0.954	36.66/0.954	37.00/0.955	37.13/0.956
Set5	×3	30.39/0.868	32.58/0.908	32.75/0.909	33.16/0.914	33.12/0.912
	×4	28.42/0.810	30.28/0.860	30.49/0.862	30.70/0.865	30.74/0.865
Set14	×2	30.24/0.868	32.28/0.905	32.42/0.906	32.63/0.909	32.72/0.917
	×3	27.55/0.774	29.13/0.818	29.28/0.820	29.43/0.824	29.43/0.837
	×4	26.00/0.702	27.32/0.749	27.49/0.750	27.59/0.754	27.60/0.772
	×2	29.56/0.843	31.21 /0.886	31.36/0.888	31.50/0.891	31.69/0.899
BSD100	×3	27.21/0.738	28.29/0.784	28.41/0.786	28.52/0.789	28.57/0.793
	×4	25.96/0.667	26.82/0.709	26.90/0.710	26.96/0.713	27.07/0.717
平均值	-	28.78/0.807	30.98/0.841	30.64/0.843	31.91/0.873	31.97/0.878

Image super-resolution reconstruction based on residual connection convolutional neural network

RICH HTML

PDF (PC)