吉林大学学报(工学版) ›› 2019, Vol. 49 ›› Issue (6): 2089-2097.doi: 10.13229/j.cnki.jdxbgxb20180637

• • 上一篇    下一篇

双通道单图像超分辨率卷积神经网络

马子骥(),卢浩,董艳茹   

  1. 湖南大学 电气与信息工程学院,长沙 410082
  • 收稿日期:2018-06-19 出版日期:2019-11-01 发布日期:2019-11-08
  • 作者简介:马子骥(1978-),男,副教授,博士.研究方向:无线通信,图像处理. E-mail:zijima@hnu.edu.cn
  • 基金资助:
    国家自然科学基金项目(61771191);教育部产学合作协同育人项目(201601004010);中央国有资本经营预算项目(财企[2013]470号);湖南省科技计划重点项目(2015JC3053);湖南省自然科学基金项目(2017JJ2052);湖南省普通高校教学改革研究项目(湘教通〔2016〕400 号);湖南省研究生创新项目(CX2017B112)

Dual path convolutional neural network forsingle image super⁃resolution

Zi-ji MA(),Hao LU,Yan-ru DONG   

  1. School of Electrical and Information Engineering, Hunan University, Changsha 410082, China
  • Received:2018-06-19 Online:2019-11-01 Published:2019-11-08

摘要:

基于残差与稠密结构的卷积神经网络的单张图像的超分辨率(SISR)方法显著地提升了重建的性能,然而残差网络侧重于特征的复用,而稠密连接可以实现对新特征的探索,为了结合两者的优势,本文设计了一种双通道图像超分辨率深度卷积神经网络,将特征映射在第三维度上分割成两条路径,一条以残差的形式进行连接,另一条以稠密的方式进行跳跃连接。同时,在网络末端引入解卷积层来放大特征映射,显著加速了计算,并且直接从低分辨率图像到高分辨率图像之间进行端到端的映射。评估结果表明,本文方案取得了比当前绝大多数网络模型更高的峰值信噪比(PSNR)。

关键词: 信息处理技术, 超分辨率, 残差网络, 稠密网络, 卷积神经网络, 峰值信噪比

Abstract:

Recent researches have shown that deep convolutional neural networks can significantly boost the performance of Single-image super-resolution (SISR). In particular, residual network and densely convolutional network can improve performance remarkably. Both path topologies are proposed to alleviate the vanishing-gradient problem of deep convolution networks. Since the residual network enables feature re-usage and the dense skip connections enables new features exploration. A dual path network is proposed for single-image super-resolution by combining the residual network and the dense skip connections in a very deep network. In the proposed network, the feature maps are split into two paths, one path is propagated in the form of residual, and another path is propagated by dense skip connections. In addition, the deconvolution layers are integrated into the network to upscale the feature map which can significantly speedup the network, and the mapping is learn from the low-resolution image to the high-resolution image directly. The network is evaluated with four benchmark datasets. The simulation results demonstrate that the proposed network has much higher peak signal-to-noise ratio(PSNR), in contrast to most conventional state-of-art methods.

Key words: information processing technology, super resolution, residual network, densely convolutional network, convolutional neural network, peak signal-to-noise ratio(PSNR)

中图分类号: 

  • TN919.8

表1

基于卷积神经网络的超分辨率算法比较"

网络模型 输入图像 层数 网络结构
SRCNN LR+Bicubic 3
VDSR LR+Bicubic 20 残差网络
DRCN LR+Bicubic 5 循环网络
LapSRN LR 27 残差网络
SRResNet LR 50 残差网络
SRDenseNet LR 68 稠密网络
DPSR(ours) LR 60 残差+稠密网络

图1

SRCNN、DRCN、VDSR、SRDenseNet以及本文提出的双通道图像超分辨率网络(DPSR)模型结构对比"

图2

双通道网络模块"

图3

子模块分裂模式"

图4

不同大小数据集PSNR测试值"

图5

相同训练集,4×SR下,使用Set5测试,前10 000次迭代的PSNR值对比"

图6

4×SR下的图像细节比较"

表2

网络模型在标准测试集下的PSNR和SSIM测试结果"

模 型 尺度因子

Set5

PSNR/SSIM

Set14

PSNR/SSIM

BSDS100

PSNR/SSIM

Urban100

PSNR/SSIM

Bicubic 4 28.42/0.810 26.00/0.703 25.96/0.669 23.14/0.658
APlus 4 30.30/0.859 27.43/0.752 26.82/0.710 24.34/0.720
SRCNN 4 30.49/0.862 27.61/0.754 26.91/0.712 24.53/0.724
FSRCNN 4 30.71/0.865 27.70/0.756 26.97/0.714 24.61/0.727
VDSR 4 31.35/0.882 28.03/0.770 27.29/0.726 25.18/0.753
DRCN 4 31.53/0.884 28.04/0.770 27.24/0.724 25.14/0.752
LapSRN 4 31.54/0.885 28.19/0.772 27.32/0.728 25.21/0.756
SRResNet 4 32.05/0.891 28.53/0.780 27.57/0.735 26.07/0.784
SRDenseNet 4 32.02/0.893 28.50/0.778 27.53/0.734 26.05/0.782
DPSR(ours) 4 32.12/0.896 28.59/0.782 27.61/0.737 26.14/0.786
1 孙旭, 李晓光, 李嘉锋, 等 . 基于深度学习的图像超分辨率复原研究进展[J]. 自动化学报, 2017, 43(5): 697-709.
1 Sun Xu , Li Xiao-guang , Li Jia-feng , et al . Review on deep learning based image super-resolution restoration algorithms[J]. Acta Automatica Sinica, 2017, 43(5): 697-709.
2 Yang J C , Wright J , Huang T S , et al . Image super-resolution as sparse representation of raw image patches[C]∥IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Anchorage, USA, 2008: 1-8.
3 Yang J C , Wright J , Huang T S , et al . Image super-resolution via sparse representation[J]. IEEE Transactions on Image Processing, 2010, 19(11): 2861-2873.
4 卜莎莎,章毓晋 . 基于局部约束线性编码的单帧和多帧图像超分辨率重建[J]. 吉林大学学报:工学版, 2013, 43(增刊1): 365-370.
4 Bu Sha-sha , Zhang Yu-jin . Single-frame and multi-frame image super-resolution based on locality-constrained linear coding[J]. Journal of Jilin University(Engineering and Technology Edition), 2013, 43(Sup.1): 365-370.
5 Timofte R , de Smet V , van Gool L . A+: adjusted anchored neighborhood regression for fast super-resolution[C]∥Asian Conference on Computer Vision, Springer, Cham, 2014: 111-126.
6 Yang C Y , Yang M H . Fast direct super-resolution by simple functions[C]∥IEEE International Conference on Computer Vision, Sydney, NSW, Australia, 2013: 561-568.
7 Schulter S , Leistner C , Bischof H . Fast and accurate image upscaling with super-resolution forests[C]∥IEEE Conference on Computer Vision and Pattern Recognition, Boston, USA, 2015: 3791-3799.
8 Dong C , Loy C C , He K , et al . Learning a deep convolutional network for image super-resolution[C]∥European Conference on Computer Vision, Springer, Cham, 2014: 184-199.
9 Dong C , Loy C C , Tang X . Accelerating the super-resolution convolutional neural network[C]∥European Conference on Computer Vision, The Netherlands, 2016: 391-407.
10 He K , Zhang X , Ren S , et al . Deep residual learning for image recognition[C]∥IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, USA, 2016: 770-778.
11 Huang G , Liu Z , Weinberger K , et al . Densely connected convolutional networks[C]∥IEEE Conference on Computer Vision and Pattern Recognition(CVPR), Hawaii, USA, 2017: 2261-2269.
12 Chen Y , Li J , Xiao H , et al . Dual path networks[C]∥Advances in Neural Information Processing Systems, Cambridge, USA, 2017: 4470-4478.
13 Zhang Y , Tian Y , Kong Y , et al . Residual dense network for image super-resolution[C]∥IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, USA, 2018:2472-2481.
14 Huang Y , Qin M . Densely connected high order residual network for single frame image super resolution[J]. arXiv preprint arXiv: 1804.05902, 2018:1-8.
15 徐冉, 张俊格,黄凯奇 . 利用双通道卷积神经网络的图像超分辨率算法[J]. 中国图象图形学报, 2016, 21(5): 556-564.
15 Xu Ran , Zhang Jun-ge , Huang Kai-qi . Image super-resolution using two-channel convolutional neural networks[J]. Journal of Image and Graphics, 2016, 21(5): 556-564.
16 Kim J , Kwon Lee J , Mu Lee K . Accurate image super-resolution using very deep convolutional networks[C]∥Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, USA, 2016: 1646-1654.
17 Lai W S , Huang J B , Ahuja N , et al . Deep laplacian pyramid networks for fast and accurate super-resolution[C]∥IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Hawaii, USA, 2017: 5835-5843.
18 Tong T , Li G , Liu X , et al . Image super-resolution using dense skip connections[C]∥IEEE International Conference on Computer Vision (ICCV), Venice, Italy, 2017: 4809-4817.
19 Yang J , Wright J , Huang T S , et al . Image super-resolution via sparse representation[J]. IEEE Transactions on Image Processing, 2010, 19(11): 2861-2873.
20 Timofte R , Agustsson E , van Gool L , et al . Ntire 2017 challenge on single image super-resolution: Methods and results[C]∥IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Honolulu, USA, 2017:1110-1121.
21 He K , Zhang X , Ren S , et al . Delving deep into rectifiers: Surpassing human-level performance on imagenet classification[C]∥IEEE International Conference on Computer Vision,Santiago, Chile, 2015: 1026-1034.
[1] 车翔玖,刘华罗,邵庆彬. 基于Fast RCNN改进的布匹瑕疵识别算法[J]. 吉林大学学报(工学版), 2019, 49(6): 2038-2044.
[2] 周柚,杨森,李大琳,吴春国,王岩,王康平. 基于现场可编程门电路的人脸检测识别加速平台[J]. 吉林大学学报(工学版), 2019, 49(6): 2051-2057.
[3] 赵宏伟,王鹏,范丽丽,胡黄水,刘萍萍. 相似性保持实例检索方法[J]. 吉林大学学报(工学版), 2019, 49(6): 2045-2050.
[4] 刘富, 权美静, 王柯, 刘云, 康冰, 韩志武, 侯涛. 仿蝎子振源定位机理的位置指纹室内定位方法[J]. 吉林大学学报(工学版), 2019, 49(6): 2076-2082.
[5] 于晓辉,张志成,李新波,孙晓东. 基于状态空间模型的指数衰减正弦信号参数估计[J]. 吉林大学学报(工学版), 2019, 49(6): 2083-2088.
[6] 卢洋,王世刚,赵文婷,赵岩. 基于离散Shearlet类别可分性测度的人脸表情识别方法[J]. 吉林大学学报(工学版), 2019, 49(5): 1715-1725.
[7] 郭继昌,吴洁,郭春乐,朱明辉. 基于残差连接卷积神经网络的图像超分辨率重构[J]. 吉林大学学报(工学版), 2019, 49(5): 1726-1734.
[8] 曹运合,曾丽,王宇. 基于特征空间的子阵级自适应和差波束测角方法[J]. 吉林大学学报(工学版), 2019, 49(5): 1735-1744.
[9] 董超,刘晶红,徐芳,王仁浩. 光学遥感图像舰船目标快速检测方法[J]. 吉林大学学报(工学版), 2019, 49(4): 1369-1376.
[10] 王柯俨,胡妍,王怀,李云松. 结合天空分割和超像素级暗通道的图像去雾算法[J]. 吉林大学学报(工学版), 2019, 49(4): 1377-1384.
[11] 托乎提努尔,张海龙,王杰,王娜,冶鑫晨,王万琼. 基于图形处理器的高速中值滤波算法[J]. 吉林大学学报(工学版), 2019, 49(3): 979-985.
[12] 付银娟,李勇,徐丽琴,张昆辉. NLFM⁃Costas射频隐身雷达信号设计及分析[J]. 吉林大学学报(工学版), 2019, 49(3): 994-999.
[13] 苏寒松,代志涛,刘高华,张倩芳. 结合吸收Markov链和流行排序的显著性区域检测[J]. 吉林大学学报(工学版), 2018, 48(6): 1887-1894.
[14] 徐岩,孙美双. 基于卷积神经网络的水下图像增强方法[J]. 吉林大学学报(工学版), 2018, 48(6): 1895-1903.
[15] 李居朋,张祖成,李墨羽,缪德芳. 基于Kalman滤波的电容屏触控轨迹平滑算法[J]. 吉林大学学报(工学版), 2018, 48(6): 1910-1916.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
[1] 王建华,王云成,付铁军,张宝生. 装用机械摩擦片式限滑差速器后轮驱动车辆的动力性[J]. 吉林大学学报(工学版), 2006, 36(02): 161 -0165 .
[2] 陈松林,姚郁,张瑞. 线性切换系统基于观测器的切换镇定[J]. 吉林大学学报(工学版), 2006, 36(01): 87 -0091 .
[3] 张英朝,傅立敏. 简单外形汽车隧道中会车过程的瞬态空气动力学数值模拟[J]. 吉林大学学报(工学版), 2006, 36(03): 302 -0306 .
[4] 刘久富,杨忠,孙德敏,李志宇. 基于模型的飞行控制软件测试用例的生成[J]. 吉林大学学报(工学版), 2006, 36(04): 543 -547 .
[5] 刘耀东, 李光玉, 连建设 . 铝掺杂氧化锌薄膜的电学及光学性能[J]. 吉林大学学报(工学版), 2007, 37(03): 495 -0498 .
[6] 宋雨来,刘耀辉,朱先勇,王素环,于思荣. 钕对AZ91镁合金组织及机械性能的影响[J]. 吉林大学学报(工学版), 2006, 36(03): 289 -0293 .
[7] 刘险峰,邹积岩,李立伟 . 大容量蓄电池组的连接方式[J]. 吉林大学学报(工学版), 2007, 37(03): 672 -0674 .
[8] 王占山 , , 张化光. 多时变时滞神经网络的全局指数稳定[J]. 吉林大学学报(工学版), 2005, 35(06): 621 -0625 .
[9] 王勇,葛建华,聂远飞 . 相位噪声条件下OFDM系统参数的优化设计
[J]. 吉林大学学报(工学版), 2008, 38(增刊): 172 -0176 .
[10] 陈塑寰,裴春艳 . 不确定二阶振动控制系统动力响应的区间方法[J]. 吉林大学学报(工学版), 2008, 38(01): 94 -98 .