Journal of Jilin University(Engineering and Technology Edition) ›› 2023, Vol. 53 ›› Issue (10): 2923-2931.doi: 10.13229/j.cnki.jdxbgxb.20211299

Previous Articles     Next Articles

Multi-scale generative adversarial network for image compressed sensing and reconstruction algorithm

Chun-yan ZENG1(),Kang YAN1,Zhi-feng WANG2(),Zheng-hui WANG1   

  1. 1.Hubei Key Laboratory for High-efficiency Utilization of Solar Energy and Operation Control of Energy Storage System，Hubei University of Technology，Wuhan 430068，China
    2.School of Education Information Technology，Central China Normal University，Wuhan 430079，China
  • Received:2021-11-29 Online:2023-10-01 Published:2023-12-13
  • Contact: Zhi-feng WANG E-mail:cyzeng@hbut.edu.cn;zfwang@mail.ccnu.edu.cn

RICH HTML

 1   

PDF (PC)

159

Abstract:

The current deep learning-based compressed sensing and reconstruction network mainly has the following problems： the single-channel reconstruction network does not deeply explore the multi-scale features of the image， lacks a feedback mechanism for the reconstruction network， and the reconstruction network lacks correlation with the measurement matrix， which restricts the further improvement of the reconstruction quality. Therefore， an image compressed sensing and reconstruction algorithm based on multi-scale generation adversarial network is proposed， which extracts the multi-scale information of images through multi-channel residual blocks， joins the discriminant network to form a feedback to the multi-scale generative network， and then trains the full convolutional measurement network jointly with the reconstruction network to improve the image reconstruction quality. Experimental results show that the reconstruction accuracy of the proposed method is improved by 2.02-4.09 dB compared with ISTA-Net+ method under three sampling rates.

Key words: computer application, compressed sensing, residual network, generative adversarial network

CLC Number: 

  • TN911.73

Fig.1

Framework of multi-scale generative adversarial network for image compressive sensing and reconstruction"

Fig.2

Fully convolutional measurement"

Fig.3

Multi-scale generative network"

Fig.4

Discriminative Network Model"

Table 1

PSNR/SSIM values of different algorithms under three sampling rates"

图 像方 法3种采样率的PSNR（dB）/SSIM图 像方 法3种采样率的PSNR（dB）/SSIM
0.100.040.010.100.040.01
BarbaraReconNet［4］21.89/0.573220.38/0.384018.61/0.3712BoatsReconNet［4］24.15/0.645421.36/0.503818.49/0.4063
DR2-Net［5］22.69/0.573220.70/0.484218.65/0.3716DR2-Net［5］25.58/0.656422.11/0.529918.67/0.4064
ISTA-Net［9］23.51/0.688420.99/0.527118.38/0.3612ISTA-Net［9］27.27/0.791422.14/0.585318.47/0.3957
ISTA-Net+［9］23.52/0.694021.12/0.537518.53/0.3698ISTA-Net+［9］27.41/0.797122.44/0.608718.51/0.4023
本 文25.12/0.748824.33/0.575222.16/0.4064本 文29.75/0.843326.79/0.708722.44/0.4805
FingerprintReconNet［4］20.75/0.700716.91/0.388314.82/0.1555ParrotReconNet［4］22.63/0.714220.27/0.652117.63/0.5324
DR2-Net［5］22.03/0.763117.40/0.453914.73/0.1554DR2-Net［5］23.94/0.742521.16/0.685918.01/0.5336
ISTA-Net［9］22.45/0.787617.31/0.445714.78/0.1501ISTA-Net［9］26.21/0.838822.24/0.699617.90/0.5302
ISTA-Net+［9］22.55/0.792217.47/0.476214.85/0.1589ISTA-Net+［9］26.37/0.852322.42/0.727218.06/0.5354
本 文27.52/0.882821.82/0.541816.75/0.1726本 文27.73/0.887325.72/0.808822.95/0.5802
LenaReconNet［4］23.83/0.710121.28/0.606117.87/0.4393ForemanReconNet［4］27.09/0.733523.72/0.596520.04/ 0.571
DR2-Net［5］25.39/0.756422.13/0.621117.97/0.4390DR2-Net［5］29.20/0.769925.34/0.681120.59/ 0.581
ISTA-Net［9］27.44/0.816722.40/0.626718.29/0.4498ISTA-Net［9］32.78/0.887325.76/0.745120.21/0.5676
ISTA-Net+［9］27.50/0.821323.02/0.663818.54/0.4557ISTA-Net+［9］33.49/0.899627.07/0.780420.34/0.5691
本 文29.47/0.864426.83/0.720622.97/0.5145本 文34.94/0.919231.97/0.841327.35/0.6542
MonarchReconNet［4］21.10/0.709418.19/0.507515.39/0.3767HouseReconNet［4］26.69/0.723322.58/0.532619.31/0.5174
DR2-Net［5］23.10/0.741418.93/0.533215.33/0.3721DR2-Net［5］27.53/0.753323.92/0.614719.61/0.5286
ISTA-Net［9］25.58/0.830419.40/0.603014.99/0.3651ISTA-Net［9］30.13/0.824024.03/0.668619.80/0.5340
ISTA-Net+［9］25.72/0.836619.56/0.630215.01/0.3641ISTA-Net+［9］30.49/0.830924.81/0.705720.00/0.5411
本 文28.55/0.883924.51/0.701718.62/0.4116本 文33.02/0.869429.71/0.742124.76/0.5832
FlinstonesReconNet［4］18.92/0.609916.30/0.442013.96/0.2475PeppersReconNet［4］22.15/0.745119.56/0.525816.82/0.4223
DR2-Net［5］21.09/0.672316.93/0.460614.01/0.2499DR2-Net［5］23.73/0.765820.32/0.534216.90/0.4226
ISTA-Net［9］23.39/0.744117.43/0.465114.00/0.2497ISTA-Net［9］26.66/0.799420.64/0.594116.88/0.4227
ISTA-Net+［9］23.39/0.751117.65/0.495513.90/0.2454ISTA-Net+［9］27.13/0.810721.10/0.617716.94/0.4255
本 文23.94/0.767320.78/0.521616.97/0.2696本 文27.58/0.834324.97/0.643921.23/0.417
CameramanReconNet［4］21.28/0.678419.26/0.583617.11/0.4619MeanReconNet［4］22.68/0.685719.99/0.520217.27/0.4092
DR2-Net［5］22.46/0.712519.84/0.593117.08/0.4588DR2-Net［5］24.32/0.718820.80/0.562917.44/0.4108
ISTA-Net［9］23.46/0.749020.27/0.597117.26/0.4643ISTA-Net［9］26.26/0.796121.15/0.596117.36/0.4082
ISTA-Net+［9］23.76/0.753920.50/0.621517.32/0.4763ISTA-Net+［9］26.48/0.803621.56/0.624017.45/0.4131
本 文25.89/0.794723.34/0.651020.77/0.4998本 文28.50/0.845025.52/0.677821.54/0.4536

Fig.5

Reconstructed visual effects of different methods at three sampling rates"

Table 2

PSNR comparison among MSRNet［6］，only multi-scale generation network and proposed method"

模型3种采样率的平均PSNR
0.100.040.01
MSRNet［6］25.1621.4117.54
仅多尺度生成网络28.0125.1321.27
本文生成对抗网络28.5025.5221.54

Table 3

Comparison of PSNR and SSIM of generation network at three scales"

测试集采样率单通道双通道三通道
PSNR/SSIMPSNR/SSIMPSNR/SSIM
Set50.0123.74/0.621924.05/0.639124.15/0.6453
0.0427.89/0.781628.54/0.812728.58/0.8136
0.1031.10/0.870931.64/0.886731.75/0.8892
Set140.0122.47/0.540822.74/0.556822.79/0.5612
0.0425.57/0.675826.09/0.698226.07/0.6989
0.1027.96/0.786028.36/0.798128.47/0.8006

Table 4

Comparison of PSNR with and without attention mechanism module in generation network"

模型3种采样率的平均PSNR
0.100.040.01
无注意力机制26.4323.9620.22
有注意力机制28.0125.1321.27
1 Donoho D. Compressed sensing[J]. IEEE Transactions on Information Theory, 2006, 52(4): 1289-1306.
2 Shi W, Jiang F, Liu S, et al. Image compressed sensing using convolutional neural network[J]. IEEE Transactions on Image Processing, 2020, 29: 375-388.
3 Mousavi A, Patel A B, Baraniuk R G. A deep learning approach to structured signal recovery[C]∥The 53rd Annual Allerton Conference on Communication, Control and Computing, Monticello,USA, 2015: 1336-1343.
4 Kulkarni K, Lohit S, Turaga P, et al. ReconNet: non-iterative reconstruction of images from compressively sensed measurements[C]∥IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, USA, 2016: 117-124.
5 Yao H T, Dai F, Zheng D M. DR2-Net:deep residual reconstruction network for image compressive sensing[C]∥IEEE Conference on Computer Vision and Pattern Recognition, Honolulu,USA, 2017: 451-462.
6 练秋生, 富利鹏, 陈书贞, 等. 基于多尺度残差网络的压缩感知重构算法[J]. 自动化学报, 2019, 45(11):10-19.
Lian Qiu-sheng, Fu Li-peng, Chen Shu-zhen, et al. A compressed sensing algorithm based on multi-scale residual reconstruction network[J]. Acta Automatica Sinica, 2019, 45(11): 10-19.
7 Gunning D, Aha D W. Darpa's explainable artificial intelligence program[J]. Magazine, 2019, 40(2): 44-58.
8 Monga V, Li Y, Eldar Y C. Algorithm unrolling: interpretable, efficient deep learning for signal and image processing[J]. IEEE Signal Processing Magazine, 2021, 38(2): 18-44.
9 Zhang J. ISTANet: interpretable optimization-inspired deep network for image compressive sensing[C]∥IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, USA, 2018: 1828-1837.
10 Fergus R, Taylor G W. Adaptive deconvolutional networks for mid and high level feature learning[C]∥International Conference on Computer Vision of IEEE Computer Society, Barcelona,Spain, 2011: 2018-2025.
11 Zeng K, Wang Y, Mao J. Deep residual deconvolutional networks for defocus blur detection[J]. IET Image Processing, 2021, 15: 1-3.
12 Woo S, Park J, Lee J Y, et al. CBAM: convolutional block attention module[C]∥Proceedings of the European Conference on Computer Vision, Munich, Germany, 2018: 3-19..
13 Hou Q, Zhou D, Feng J. Coordinate attention for efficient mobile network design[C]∥Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Online, 2021: 13713-13722.
14 Hu A, Chen S, Wu L. WSGAN: an improved generative adversarial network for remote sensing image road network extraction by weakly supervised processing[J]. Remote Sensing, 2021, 13(13): 2506-2516.
15 Cui T, Gou G, Xiong G.6GAN: IPv6 multi-pattern target generation via generative adversarial nets with reinforcement learning[C]∥IEEE Conference on Computer Communications, Beijing, China, 2021: 1-10.
16 LEDIG C. Photo-realistic single image super-resolution using a generative adversarial network[C]∥IEEE Conference on Computer Vision and Pattern Recognition, Honolulu,USA, 2017: 105-114.
17 Deng Z, Zhu L, Hu X. Deep multi-model fusion for single-image dehazing[C]∥International Conference on Computer Vision, Seoul, South Korea, 2019: 165-174.
[1] Guang HUO,Da-wei LIN,Yuan-ning LIU,Xiao-dong ZHU,Meng YUAN,Di GAI. Lightweight iris segmentation model based on multiscale feature and attention mechanism [J]. Journal of Jilin University(Engineering and Technology Edition), 2023, 53(9): 2591-2600.
[2] Ying HE,Zhuo-ran WANG,Xu ZHOU,Yan-heng LIU. Point of interest recommendation algorithm integrating social geographical information based on weighted matrix factorization [J]. Journal of Jilin University(Engineering and Technology Edition), 2023, 53(9): 2632-2639.
[3] Yun-zuo ZHANG,Xu DONG,Zhao-quan CAI. Multi view gait cycle detection by fitting geometric features of lower limbs [J]. Journal of Jilin University(Engineering and Technology Edition), 2023, 53(9): 2611-2619.
[4] Ming-yao XIAO,Xiong-fei LI,Rui ZHU. Medical image fusion based on pixel correlation analysis in NSST domain [J]. Journal of Jilin University(Engineering and Technology Edition), 2023, 53(9): 2640-2648.
[5] Ya-hui ZHAO,Fei-yu LI,Rong-yi CUI,Guo-zhe JIN,Zhen-guo ZHANG,De LI,Xiao-feng JIN. Korean⁃Chinese translation quality estimation based on cross⁃lingual pretraining model [J]. Journal of Jilin University(Engineering and Technology Edition), 2023, 53(8): 2371-2379.
[6] Xiang-jiu CHE,Huan XU,Ming-yang PAN,Quan-le LIU. Two-stage learning algorithm for biomedical named entity recognition [J]. Journal of Jilin University(Engineering and Technology Edition), 2023, 53(8): 2380-2387.
[7] Lian-ming WANG,Xin WU. Method for 3D motion parameter measurement based on pose estimation [J]. Journal of Jilin University(Engineering and Technology Edition), 2023, 53(7): 2099-2108.
[8] Ze-qiang ZHANG,Wei LIANG,Meng-ke XIE,Hong-bin ZHENG. Elite differential evolution algorithm for mixed⁃model two⁃side disassembly line balancing problem [J]. Journal of Jilin University(Engineering and Technology Edition), 2023, 53(5): 1297-1304.
[9] Zhen-hai ZHANG,Kun JI,Jian-wu DANG. Crack identification method for bridge based on BCEM model [J]. Journal of Jilin University(Engineering and Technology Edition), 2023, 53(5): 1418-1426.
[10] Pei-yong LIU,Jie DONG,Luo-feng XIE,Yang-yang ZHU,Guo-fu YIN. Surface defect detection algorithm of magnetic tiles based on multi⁃branch convolutional neural network [J]. Journal of Jilin University(Engineering and Technology Edition), 2023, 53(5): 1449-1457.
[11] Yu JIANG,Jia-zheng PAN,He-huai CHEN,Ling-zhi FU,Hong QI. Segmentation-based detector for traditional Chinese newspaper [J]. Journal of Jilin University(Engineering and Technology Edition), 2023, 53(4): 1146-1154.
[12] Wei LYU,Jia-ze HAN,Jing-hui CHU,Pei-guang JING. Multi⁃modal self⁃attention network for video memorability prediction [J]. Journal of Jilin University(Engineering and Technology Edition), 2023, 53(4): 1211-1219.
[13] Peng YU,Yan PIAO. New method for extracting person re-identification attributes based on multi-scale features [J]. Journal of Jilin University(Engineering and Technology Edition), 2023, 53(4): 1155-1162.
[14] Hong-yang PAN,Zhao LIU,Bo YANG,Geng SUN,Yan-heng LIU. Overview of swarm intelligence methods for unmanned aerial vehicle systems based on new⁃generation information technology [J]. Journal of Jilin University(Engineering and Technology Edition), 2023, 53(3): 629-642.
[15] Ying HE,Jun-song FAN,Wei WANG,Geng SUN,Yan-heng LIU. Joint optimization of secure communication and trajectory planning in unmanned aerial vehicle air⁃to⁃ground [J]. Journal of Jilin University(Engineering and Technology Edition), 2023, 53(3): 913-922.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
Copyright © Journal of Jilin University(Engineering and Technology Edition), All Rights Reserved.
Tel: +86-431-85095297　Fax: +86-431-85094074　E-mail: xbgxb@jlu.edu.cn
Powered by Beijing Magtech Co. Ltd