基于残差连接卷积神经网络的图像超分辨率重构

doi:10.13229/j.cnki.jdxbgxb20180366

摘要/Abstract

摘要：

针对基于卷积神经网络超分辨率重构算法中存在的感受野较小、梯度信息易丢失与网络收敛较慢等问题，提出了基于残差连接卷积神经网络的图像超分辨率重构算法。通过在低分辨率空间进行图像的超分辨率重构，减少了图像预处理过程，降低了网络复杂度。利用局部和全局残差连接，对卷积网络结构和亚像素采样层进行改进，局部残差促进了网络中信息的流动，全局残差使网络只学习图像残差信息，减少了网络冗余。通过增加网络深度扩大了感受野，使网络学习到更多的重建信息。实验结果表明：本文算法的PSNR和SSIM值相较于其他算法有不同程度的提升。

关键词: 信息处理技术, 图像超分辨率重构, 卷积神经网络, 亚像素采样层, 残差连接

Abstract:

Super-resolution reconstruction algorithms based on convolutional neural network are confront the problems of small receptive field, disappearing of gradient information and slow convergence of the network. In order to solve these problems, an image super-resolution reconstruction algorithm based on residual connection convolutional neural network is proposed. Processing images in high resolution space results in the increase in network complexity. So, images are processed in low-resolution space to reduce complexity. The convolutional neural network is improved by introducing local and global residuals. Local residual promotes the flow of information and avoids the problem of gradient disappearing during network training. Due to the local residual connection, image information can be transferred to deeper layers. The global residual makes the network learn the residual information in images only, reducing network redundancy and accelerating network convergence. The receptive field is expanded by increasing the network depth, which makes the network learn more reconstruction information. The PSNR and SSIM of the proposed algorithm are comparable to other algorithms.

Key words: imformation processing technology, image super-resolution, convolution neural network, subpixel upsampling, residual connection

中图分类号:

TP751.1

郭继昌,吴洁,郭春乐,朱明辉. 基于残差连接卷积神经网络的图像超分辨率重构[J]. 吉林大学学报(工学版), 2019, 49(5): 1726-1734.

Ji-chang GUO,Jie WU,Chun-le GUO,Ming-hui ZHU. Image super-resolution reconstruction based on residual connection convolutional neural network[J]. Journal of Jilin University(Engineering and Technology Edition), 2019, 49(5): 1726-1734.

图/表 12

图1

图2

图3

图4

表1

图5

表2

图6

图7

图8

图9

表3

参考文献 19

1	Tsai R Y , Huang T S . Multiframe image restoration and registration[J]. Advance Computer Visual and Image Processing, 1984, 1(2): 317-339.
2	Peleg S , Keren D , Schweitzer L . Improving image resolution using subpixel motion[J]. Pattern Recognition Letters, 1987, 5(3): 223-226.
3	Schultz R R , Stevenson R L . Extraction of high-resolution frames from video sequences[J]. IEEE Transactions on Image Processing, 1996, 5(6): 996-1011.
4	Freeman W T , Jones T R , Pasztor E C . Example-based super-resolution[J]. IEEE Computer Graphics & Applications, 2002, 22(2): 56-65.
5	薛翠红，于明，杨宇皓, 等 . 基于学习的马尔科夫超分辨率复原[J]. 吉林大学学报: 工学版， 2013, 43(增刊1): 406-409.
	Xue Cui-hong , Yu Ming , Yang Yu-hao , et al . MRF reconstruction based on Markov network[J]. Journal of Jilin University (Engineering and Technology Edition), 2013, 43(Sup.1): 406-409.
6	徐岩, 孙美双 . 基于卷积神经网络的水下图像增强方法[J]. 吉林大学学报：工学版, 2018, 48(6): 1895-1903.
	Xu Yan , Sun Mei-shuang . Enhancing underwater image based on convolutional neural networks[J]. Journal of Jilin University (Engineering and Technology Edition), 2018, 48(6): 1895-1903.
7	马淼, 李贻斌 . 基于多级图像序列和卷积神经网络的人体行为识别[J]. 吉林大学学报：工学版, 2017, 47(4): 1244-1252.
	Ma Miao , Li Yi-bin . Multi-level image sequences and convolutional neural networks based human action recognition method[J]. Journal of Jilin University (Engineering and Technology Edition)， 2017, 47(4): 1244-1252.
8	Dong C , Chen C L , He K , et al . Learning a deep convolutional network for image super-resolution[C]∥European Conference on Computer Vision. Cham: Springer International Publishing, 2014: 184-199.
9	Dong C , Loy C C , Tang X . Accelerating the super-resolution convolutional neural network[C]∥European Conference on Computer Vision. Amsterdam: Springer International Publishing, 2016: 391-407.
10	Shi W , Caballero J , Huszár F , et al . Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network[C]∥IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas: IEEE Computer Society, 2016: 1874-1883.
11	Kim J , Lee J K , Lee K M . Accurate image super-resolution using very deep convolutional networks[C]∥IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas: IEEE Computer Society, 2016: 1646-1654.
12	Kim J , Kwon Lee J , Mu Lee K . Deeply-recursive convolutional network for image super-resolution[C]∥IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas: IEEE Computer Society, 2016: 1637-1645.
13	Justin J , Alexandre A , Li F F . Perceptual losses for real-time style transfer and super-resolution[C]∥European Conference on Computer Vision. Amsterdam: Springer International Publishing, 2016: 694-711.
14	Ledig C , Wang Z , Shi W , et al . Photo-realistic single image super-resolution using a generative adversarial network[C]∥IEEE Conference on Computer Vision and Pattern Recognition. Hawaii: IEEE Computer Society, 2017: 105-114.
15	Cui Z , Chang H , Shan S , et al . Deep network cascade for image super-resolution[C]∥European Conference on Computer Vision. Cham: Springer International Publishing, 2014: 49-64.
16	Mao X J , Shen C , Yang Y B . Image restoration using very deep convolutional encoder-decoder networks with symmetric skip connections[J]. Advances in Neural Information Processing Systems, 2016(2): 2810-2818.
17	Wang Y F , Wang L J , Wang H Y , et al . End-to-end image super-resolution via deep and shallow convolutional networks[J]. IEEE Access, 2016(7): 959-970.
18	He K , Zhang X , Ren S , et al . Deep residual learning for image recognition[C]∥IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas: IEEE Computer Society, 2016: 770-778.
19	Nah S , Kim T H , Lee K M . Deep multi-scale convolutional neural network for dynamic scene deblurring[C]∥IEEE Conference on Computer Vision and Pattern Recognition. Hawaii: IEEE Computer Society, 2017: 257-265.

相关文章 15

[1]	卢洋,王世刚,赵文婷,赵岩. 基于离散Shearlet类别可分性测度的人脸表情识别方法[J]. 吉林大学学报(工学版), 2019, 49(5): 1715-1725.
[2]	董超,刘晶红,徐芳,王仁浩. 光学遥感图像舰船目标快速检测方法[J]. 吉林大学学报(工学版), 2019, 49(4): 1369-1376.
[3]	王柯俨,胡妍,王怀,李云松. 结合天空分割和超像素级暗通道的图像去雾算法[J]. 吉林大学学报(工学版), 2019, 49(4): 1377-1384.
[4]	托乎提努尔,张海龙,王杰,王娜,冶鑫晨,王万琼. 基于图形处理器的高速中值滤波算法[J]. 吉林大学学报(工学版), 2019, 49(3): 979-985.
[5]	付银娟,李勇,徐丽琴,张昆辉. NLFM⁃Costas射频隐身雷达信号设计及分析[J]. 吉林大学学报(工学版), 2019, 49(3): 994-999.
[6]	苏寒松,代志涛,刘高华,张倩芳. 结合吸收Markov链和流行排序的显著性区域检测[J]. 吉林大学学报(工学版), 2018, 48(6): 1887-1894.
[7]	徐岩,孙美双. 基于卷积神经网络的水下图像增强方法[J]. 吉林大学学报(工学版), 2018, 48(6): 1895-1903.
[8]	李居朋,张祖成,李墨羽,缪德芳. 基于Kalman滤波的电容屏触控轨迹平滑算法[J]. 吉林大学学报(工学版), 2018, 48(6): 1910-1916.
[9]	黄勇,杨德运,乔赛,慕振国. 高分辨合成孔径雷达图像的耦合传统恒虚警目标检测[J]. 吉林大学学报(工学版), 2018, 48(6): 1904-1909.
[10]	应欢,刘松华,唐博文,韩丽芳,周亮. 基于自适应释放策略的低开销确定性重放方法[J]. 吉林大学学报(工学版), 2018, 48(6): 1917-1924.
[11]	陆智俊,钟超,吴敬玉. 星载合成孔径雷达图像小特征的准确分割方法[J]. 吉林大学学报(工学版), 2018, 48(6): 1925-1930.
[12]	刘仲民,王阳,李战明,胡文瑾. 基于简单线性迭代聚类和快速最近邻区域合并的图像分割算法[J]. 吉林大学学报(工学版), 2018, 48(6): 1931-1937.
[13]	单泽彪,刘小松,史红伟,王春阳,石要武. 动态压缩感知波达方向跟踪算法[J]. 吉林大学学报(工学版), 2018, 48(6): 1938-1944.
[14]	姚海洋, 王海燕, 张之琛, 申晓红. 双Duffing振子逆向联合信号检测模型[J]. 吉林大学学报(工学版), 2018, 48(4): 1282-1290.
[15]	全薇, 郝晓明, 孙雅东, 柏葆华, 王禹亭. 基于实际眼结构的个性化投影式头盔物镜研制[J]. 吉林大学学报(工学版), 2018, 48(4): 1291-1297.

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed

数据库	尺度因子	双三次插值PSNR/SSIM	A+ PSNR/SSIM	SRCNN PSNR/SSIM	FSRCNN PSNR/SSIM	本文方法PSNR/SSIM
Set5	×2	33.66/0.992	36.54/0.954	36.66/0.954	37.00/0.955	37.13/0.956
Set5	×3	30.39/0.868	32.58/0.908	32.75/0.909	33.16/0.914	33.12/0.912
	×4	28.42/0.810	30.28/0.860	30.49/0.862	30.70/0.865	30.74/0.865
Set14	×2	30.24/0.868	32.28/0.905	32.42/0.906	32.63/0.909	32.72/0.917
	×3	27.55/0.774	29.13/0.818	29.28/0.820	29.43/0.824	29.43/0.837
	×4	26.00/0.702	27.32/0.749	27.49/0.750	27.59/0.754	27.60/0.772
	×2	29.56/0.843	31.21 /0.886	31.36/0.888	31.50/0.891	31.69/0.899
BSD100	×3	27.21/0.738	28.29/0.784	28.41/0.786	28.52/0.789	28.57/0.793
	×4	25.96/0.667	26.82/0.709	26.90/0.710	26.96/0.713	27.07/0.717
平均值	-	28.78/0.807	30.98/0.841	30.64/0.843	31.91/0.873	31.97/0.878