Journal of Jilin University(Engineering and Technology Edition) ›› 2019, Vol. 49 ›› Issue (6): 2089-2097.doi: 10.13229/j.cnki.jdxbgxb20180637

Previous Articles     Next Articles

Dual path convolutional neural network forsingle image super⁃resolution

Zi-ji MA(),Hao LU,Yan-ru DONG   

  1. School of Electrical and Information Engineering, Hunan University, Changsha 410082, China
  • Received:2018-06-19 Online:2019-11-01 Published:2019-11-08

Abstract:

Recent researches have shown that deep convolutional neural networks can significantly boost the performance of Single-image super-resolution (SISR). In particular, residual network and densely convolutional network can improve performance remarkably. Both path topologies are proposed to alleviate the vanishing-gradient problem of deep convolution networks. Since the residual network enables feature re-usage and the dense skip connections enables new features exploration. A dual path network is proposed for single-image super-resolution by combining the residual network and the dense skip connections in a very deep network. In the proposed network, the feature maps are split into two paths, one path is propagated in the form of residual, and another path is propagated by dense skip connections. In addition, the deconvolution layers are integrated into the network to upscale the feature map which can significantly speedup the network, and the mapping is learn from the low-resolution image to the high-resolution image directly. The network is evaluated with four benchmark datasets. The simulation results demonstrate that the proposed network has much higher peak signal-to-noise ratio(PSNR), in contrast to most conventional state-of-art methods.

Key words: information processing technology, super resolution, residual network, densely convolutional network, convolutional neural network, peak signal-to-noise ratio(PSNR)

CLC Number: 

  • TN919.8

Table 1

Comparisons of CNN based SR algorithms"

网络模型 输入图像 层数 网络结构
SRCNN LR+Bicubic 3
VDSR LR+Bicubic 20 残差网络
DRCN LR+Bicubic 5 循环网络
LapSRN LR 27 残差网络
SRResNet LR 50 残差网络
SRDenseNet LR 68 稠密网络
DPSR(ours) LR 60 残差+稠密网络

Fig.1

Network architectures of SRCNN, DRCN, VDSR, SRDenseNet and proposed dual path network for single image super resolution(DPSR)"

Fig.2

Structure of dual path blocks"

Fig.3

Split models of micro-blocks"

"

Fig.5

PSNR convergence curve on Set5 for 4×SR of first 10 000 iterations with the same training set"

Fig.6

Visual comparison for 4× SR"

Table 2

Comparison of SR results in terms of PSNR/SSIM using four benchmark datesets"

模 型 尺度因子

Set5

PSNR/SSIM

Set14

PSNR/SSIM

BSDS100

PSNR/SSIM

Urban100

PSNR/SSIM

Bicubic 4 28.42/0.810 26.00/0.703 25.96/0.669 23.14/0.658
APlus 4 30.30/0.859 27.43/0.752 26.82/0.710 24.34/0.720
SRCNN 4 30.49/0.862 27.61/0.754 26.91/0.712 24.53/0.724
FSRCNN 4 30.71/0.865 27.70/0.756 26.97/0.714 24.61/0.727
VDSR 4 31.35/0.882 28.03/0.770 27.29/0.726 25.18/0.753
DRCN 4 31.53/0.884 28.04/0.770 27.24/0.724 25.14/0.752
LapSRN 4 31.54/0.885 28.19/0.772 27.32/0.728 25.21/0.756
SRResNet 4 32.05/0.891 28.53/0.780 27.57/0.735 26.07/0.784
SRDenseNet 4 32.02/0.893 28.50/0.778 27.53/0.734 26.05/0.782
DPSR(ours) 4 32.12/0.896 28.59/0.782 27.61/0.737 26.14/0.786
1 孙旭, 李晓光, 李嘉锋, 等 . 基于深度学习的图像超分辨率复原研究进展[J]. 自动化学报, 2017, 43(5): 697-709.
1 Sun Xu , Li Xiao-guang , Li Jia-feng , et al . Review on deep learning based image super-resolution restoration algorithms[J]. Acta Automatica Sinica, 2017, 43(5): 697-709.
2 Yang J C , Wright J , Huang T S , et al . Image super-resolution as sparse representation of raw image patches[C]∥IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Anchorage, USA, 2008: 1-8.
3 Yang J C , Wright J , Huang T S , et al . Image super-resolution via sparse representation[J]. IEEE Transactions on Image Processing, 2010, 19(11): 2861-2873.
4 卜莎莎,章毓晋 . 基于局部约束线性编码的单帧和多帧图像超分辨率重建[J]. 吉林大学学报:工学版, 2013, 43(增刊1): 365-370.
4 Bu Sha-sha , Zhang Yu-jin . Single-frame and multi-frame image super-resolution based on locality-constrained linear coding[J]. Journal of Jilin University(Engineering and Technology Edition), 2013, 43(Sup.1): 365-370.
5 Timofte R , de Smet V , van Gool L . A+: adjusted anchored neighborhood regression for fast super-resolution[C]∥Asian Conference on Computer Vision, Springer, Cham, 2014: 111-126.
6 Yang C Y , Yang M H . Fast direct super-resolution by simple functions[C]∥IEEE International Conference on Computer Vision, Sydney, NSW, Australia, 2013: 561-568.
7 Schulter S , Leistner C , Bischof H . Fast and accurate image upscaling with super-resolution forests[C]∥IEEE Conference on Computer Vision and Pattern Recognition, Boston, USA, 2015: 3791-3799.
8 Dong C , Loy C C , He K , et al . Learning a deep convolutional network for image super-resolution[C]∥European Conference on Computer Vision, Springer, Cham, 2014: 184-199.
9 Dong C , Loy C C , Tang X . Accelerating the super-resolution convolutional neural network[C]∥European Conference on Computer Vision, The Netherlands, 2016: 391-407.
10 He K , Zhang X , Ren S , et al . Deep residual learning for image recognition[C]∥IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, USA, 2016: 770-778.
11 Huang G , Liu Z , Weinberger K , et al . Densely connected convolutional networks[C]∥IEEE Conference on Computer Vision and Pattern Recognition(CVPR), Hawaii, USA, 2017: 2261-2269.
12 Chen Y , Li J , Xiao H , et al . Dual path networks[C]∥Advances in Neural Information Processing Systems, Cambridge, USA, 2017: 4470-4478.
13 Zhang Y , Tian Y , Kong Y , et al . Residual dense network for image super-resolution[C]∥IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, USA, 2018:2472-2481.
14 Huang Y , Qin M . Densely connected high order residual network for single frame image super resolution[J]. arXiv preprint arXiv: 1804.05902, 2018:1-8.
15 徐冉, 张俊格,黄凯奇 . 利用双通道卷积神经网络的图像超分辨率算法[J]. 中国图象图形学报, 2016, 21(5): 556-564.
15 Xu Ran , Zhang Jun-ge , Huang Kai-qi . Image super-resolution using two-channel convolutional neural networks[J]. Journal of Image and Graphics, 2016, 21(5): 556-564.
16 Kim J , Kwon Lee J , Mu Lee K . Accurate image super-resolution using very deep convolutional networks[C]∥Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, USA, 2016: 1646-1654.
17 Lai W S , Huang J B , Ahuja N , et al . Deep laplacian pyramid networks for fast and accurate super-resolution[C]∥IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Hawaii, USA, 2017: 5835-5843.
18 Tong T , Li G , Liu X , et al . Image super-resolution using dense skip connections[C]∥IEEE International Conference on Computer Vision (ICCV), Venice, Italy, 2017: 4809-4817.
19 Yang J , Wright J , Huang T S , et al . Image super-resolution via sparse representation[J]. IEEE Transactions on Image Processing, 2010, 19(11): 2861-2873.
20 Timofte R , Agustsson E , van Gool L , et al . Ntire 2017 challenge on single image super-resolution: Methods and results[C]∥IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Honolulu, USA, 2017:1110-1121.
21 He K , Zhang X , Ren S , et al . Delving deep into rectifiers: Surpassing human-level performance on imagenet classification[C]∥IEEE International Conference on Computer Vision,Santiago, Chile, 2015: 1026-1034.
[1] Hong-wei ZHAO,Peng WANG,Li-li FAN,Huang-shui HU,Ping-ping LIU. Similarity retention instance retrieval method [J]. Journal of Jilin University(Engineering and Technology Edition), 2019, 49(6): 2045-2050.
[2] Fu LIU, Mei-jing QUAN, Ke WANG, Yun LIU, Bing KANG, Zhi-wu HAN, Tao HOU. Indoor positioning method based on location fingerprinting of imitating mechanism of scorpion vibration source [J]. Journal of Jilin University(Engineering and Technology Edition), 2019, 49(6): 2076-2082.
[3] Xiao-hui YU,Zhi-cheng ZHANG,Xin-bo LI,Xiao-dong SUN. Parameter estimation of exponentially damped sinusoidsbased on state⁃space model [J]. Journal of Jilin University(Engineering and Technology Edition), 2019, 49(6): 2083-2088.
[4] Xiang-jiu CHE,Hua-luo LIU,Qing-bin SHAO. Fabric defect recognition algorithm based onimproved Fast RCNN [J]. Journal of Jilin University(Engineering and Technology Edition), 2019, 49(6): 2038-2044.
[5] Yang LU,Shi-gang WANG,Wen-ting ZHAO,Yan ZHAO. Facial expression recognition based on separability assessment of discrete Shearlet transform [J]. Journal of Jilin University(Engineering and Technology Edition), 2019, 49(5): 1715-1725.
[6] Chao DONG,Jing⁃hong LIU,Fang XU,Ren⁃hao WANG. Fast ship detection in optical remote sensing images [J]. Journal of Jilin University(Engineering and Technology Edition), 2019, 49(4): 1369-1376.
[7] Ke⁃yan WANG,Yan HU,Huai WANG,Yun⁃song LI. Image dehazing algorithm by sky segmentation and superpixel⁃level dark channel [J]. Journal of Jilin University(Engineering and Technology Edition), 2019, 49(4): 1377-1384.
[8] Tohtonur,Hai⁃long ZHANG,Jie WANG,Na WANG,Xin⁃chen YE,Wan⁃qiong WANG. High speed median filtering algorithm based on graphics processing unit [J]. Journal of Jilin University(Engineering and Technology Edition), 2019, 49(3): 979-985.
[9] Lu⁃tao LIU,Na LI. Source detection based on coprime array [J]. Journal of Jilin University(Engineering and Technology Edition), 2019, 49(3): 986-993.
[10] Yin⁃juan FU,Yong LI,Li⁃qin XU,Kun⁃hui ZHANG. Design and analysis of NLFM⁃Costas RF stealth radar signal [J]. Journal of Jilin University(Engineering and Technology Edition), 2019, 49(3): 994-999.
[11] XU Yan,SUN Mei-shuang. Enhancing underwater image based on convolutional neural networks [J]. Journal of Jilin University(Engineering and Technology Edition), 2018, 48(6): 1895-1903.
[12] YING Huan,LIU Song-hua,TANG Bo-wen,HAN Li-fang,ZHOU Liang. Efficient deterministic replay technique based on adaptive release strategy [J]. Journal of Jilin University(Engineering and Technology Edition), 2018, 48(6): 1917-1924.
[13] LIU Zhong-min,WANG Yang,LI Zhan-ming,HU Wen-jin. Image segmentation algorithm based on SLIC and fast nearest neighbor region merging [J]. Journal of Jilin University(Engineering and Technology Edition), 2018, 48(6): 1931-1937.
[14] SHAN Ze-biao,LIU Xiao-song,SHI Hong-wei,WANG Chun-yang,SHI Yao-wu. DOA tracking algorithm using dynamic compressed sensing [J]. Journal of Jilin University(Engineering and Technology Edition), 2018, 48(6): 1938-1944.
[15] YAO Hai-yang, WANG Hai-yan, ZHANG Zhi-chen, SHEN Xiao-hong. Reverse-joint signal detection model with double Duffing oscillator [J]. 吉林大学学报(工学版), 2018, 48(4): 1282-1290.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
[1] Wang Jianhua,Wang Yuncheng,Fu Tiejun,Zhang Baosheng. Power performance of rear wheel drive vehicle with disktype mechanical limitedslip differential[J]. 吉林大学学报(工学版), 2006, 36(02): 161 -0165 .
[2] Chen Songlin,Yao Yu,Zhang Rui. Stabilizing Switched Linear Systems via Observerbased Switching[J]. 吉林大学学报(工学版), 2006, 36(01): 87 -0091 .
[3] Zhang Ying-chao, Fu Li-min. Transient aerodynamic numerical simulation of simplified shape cars under condition of opposite meet sidebyside in tunnel[J]. 吉林大学学报(工学版), 2006, 36(03): 302 -0306 .
[4] Liu Jiu-fu,Yang Zhong,Sun De-min, Li Zhi-yu. Generation of testing case for model based flight control software of unmanned aerial vehicle[J]. 吉林大学学报(工学版), 2006, 36(04): 543 -547 .
[5] Liu Yao-dong, Li Guang-yu, Lian Jian-she. Effect of Al-doping on electrical and optical properties of ZnO films[J]. 吉林大学学报(工学版), 2007, 37(03): 495 -0498 .
[6] Song Yu-lai, Liu Yao-hui, Zhu Xian-yong, Wang Su-huan, Yu Si-rong. Effect of neodymium on microstructure and mechanical properties of AZ91 magnesium alloy[J]. 吉林大学学报(工学版), 2006, 36(03): 289 -0293 .
[7] Liu Xian-feng,Zou Ji-yan,Li Li-wei . Study on reliability of storage battery array with high capacity[J]. 吉林大学学报(工学版), 2007, 37(03): 672 -0674 .
[8] WANG Zhanshan , , ZHANG Huaguang . Exponential Stability of Neural Networks with Multiple TimeVarying Time Delays[J]. 吉林大学学报(工学版), 2005, 35(06): 621 -0625 .
[9] Wang Yong, Ge Jian-hua, Nie Yuan-fei . An optimal design criterion for parameters OFDM
systems parameters under phase noise
[J]. 吉林大学学报(工学版), 2008, 38(增刊): 172 -0176 .
[10] Chen Su-huan,Pei Chun-yan . Dynamic response of second-order uncertain vibration control systems with interval method[J]. 吉林大学学报(工学版), 2008, 38(01): 94 -98 .