吉林大学学报(工学版) ›› 2023, Vol. 53 ›› Issue (10): 2923-2931.doi: 10.13229/j.cnki.jdxbgxb.20211299

• 计算机科学与技术 • 上一篇    下一篇

多尺度生成对抗网络下图像压缩感知重建算法

曾春艳1(),严康1,王志锋2(),王正辉1   

  1. 1.湖北工业大学 太阳能高效利用及储能运行控制湖北省重点实验室,武汉 430068
    2.华中师范大学 教育信息技术学院,武汉 430079
  • 收稿日期:2021-11-29 出版日期:2023-10-01 发布日期:2023-12-13
  • 通讯作者: 王志锋 E-mail:cyzeng@hbut.edu.cn;zfwang@mail.ccnu.edu.cn
  • 作者简介:曾春艳(1986-),女,副教授,博士.研究方向:压缩感知,可解释机器学习. E-mail:cyzeng@hbut.edu.cn
  • 基金资助:
    国家自然科学基金项目(61901165);湖北省自然科学基金项目(2022CFA007);武汉市知识创新项目(2022020801010258);华中师范大学人工智能助推教师队伍建设行动试点项目(CCNUAI&FE2022-03-01)

Multi-scale generative adversarial network for image compressed sensing and reconstruction algorithm

Chun-yan ZENG1(),Kang YAN1,Zhi-feng WANG2(),Zheng-hui WANG1   

  1. 1.Hubei Key Laboratory for High-efficiency Utilization of Solar Energy and Operation Control of Energy Storage System,Hubei University of Technology,Wuhan 430068,China
    2.School of Education Information Technology,Central China Normal University,Wuhan 430079,China
  • Received:2021-11-29 Online:2023-10-01 Published:2023-12-13
  • Contact: Zhi-feng WANG E-mail:cyzeng@hbut.edu.cn;zfwang@mail.ccnu.edu.cn

摘要:

针对目前基于深度学习的压缩感知重建网络存在单通道重建网络没有深入挖掘图像的多尺度特征,缺乏对重建网络的反馈机制,并且重建网络缺乏与测量矩阵的关联,制约了重建质量的进一步提升的问题,提出了一种多尺度生成对抗网络下图像压缩感知与重建算法。该算法先通过多通道残差块提取图像的多尺度信息,加入判别网络形成对多尺度生成网络的反馈,再将全卷积测量网络与重建网络联合训练,以提升图像重建质量。实验结果表明:本文方法相对于ISTA-Net+方法在3种采样率下重建精度提高了2.02~4.09 dB。

关键词: 计算机应用, 压缩感知, 残差网络, 生成对抗网络

Abstract:

The current deep learning-based compressed sensing and reconstruction network mainly has the following problems: the single-channel reconstruction network does not deeply explore the multi-scale features of the image, lacks a feedback mechanism for the reconstruction network, and the reconstruction network lacks correlation with the measurement matrix, which restricts the further improvement of the reconstruction quality. Therefore, an image compressed sensing and reconstruction algorithm based on multi-scale generation adversarial network is proposed, which extracts the multi-scale information of images through multi-channel residual blocks, joins the discriminant network to form a feedback to the multi-scale generative network, and then trains the full convolutional measurement network jointly with the reconstruction network to improve the image reconstruction quality. Experimental results show that the reconstruction accuracy of the proposed method is improved by 2.02-4.09 dB compared with ISTA-Net+ method under three sampling rates.

Key words: computer application, compressed sensing, residual network, generative adversarial network

中图分类号: 

  • TN911.73

图1

多尺度生成对抗网络下图像压缩感知与重建框架"

图2

全卷积测量"

图3

多尺度生成网络"

图4

判别网络模型"

表1

3种采样率下不同算法的PSNR/SSIM值"

图 像方 法3种采样率的PSNR(dB)/SSIM图 像方 法3种采样率的PSNR(dB)/SSIM
0.100.040.010.100.040.01
BarbaraReconNet421.89/0.573220.38/0.384018.61/0.3712BoatsReconNet424.15/0.645421.36/0.503818.49/0.4063
DR2-Net522.69/0.573220.70/0.484218.65/0.3716DR2-Net525.58/0.656422.11/0.529918.67/0.4064
ISTA-Net923.51/0.688420.99/0.527118.38/0.3612ISTA-Net927.27/0.791422.14/0.585318.47/0.3957
ISTA-Net+923.52/0.694021.12/0.537518.53/0.3698ISTA-Net+927.41/0.797122.44/0.608718.51/0.4023
本 文25.12/0.748824.33/0.575222.16/0.4064本 文29.75/0.843326.79/0.708722.44/0.4805
FingerprintReconNet420.75/0.700716.91/0.388314.82/0.1555ParrotReconNet422.63/0.714220.27/0.652117.63/0.5324
DR2-Net522.03/0.763117.40/0.453914.73/0.1554DR2-Net523.94/0.742521.16/0.685918.01/0.5336
ISTA-Net922.45/0.787617.31/0.445714.78/0.1501ISTA-Net926.21/0.838822.24/0.699617.90/0.5302
ISTA-Net+922.55/0.792217.47/0.476214.85/0.1589ISTA-Net+926.37/0.852322.42/0.727218.06/0.5354
本 文27.52/0.882821.82/0.541816.75/0.1726本 文27.73/0.887325.72/0.808822.95/0.5802
LenaReconNet423.83/0.710121.28/0.606117.87/0.4393ForemanReconNet427.09/0.733523.72/0.596520.04/ 0.571
DR2-Net525.39/0.756422.13/0.621117.97/0.4390DR2-Net529.20/0.769925.34/0.681120.59/ 0.581
ISTA-Net927.44/0.816722.40/0.626718.29/0.4498ISTA-Net932.78/0.887325.76/0.745120.21/0.5676
ISTA-Net+927.50/0.821323.02/0.663818.54/0.4557ISTA-Net+933.49/0.899627.07/0.780420.34/0.5691
本 文29.47/0.864426.83/0.720622.97/0.5145本 文34.94/0.919231.97/0.841327.35/0.6542
MonarchReconNet421.10/0.709418.19/0.507515.39/0.3767HouseReconNet426.69/0.723322.58/0.532619.31/0.5174
DR2-Net523.10/0.741418.93/0.533215.33/0.3721DR2-Net527.53/0.753323.92/0.614719.61/0.5286
ISTA-Net925.58/0.830419.40/0.603014.99/0.3651ISTA-Net930.13/0.824024.03/0.668619.80/0.5340
ISTA-Net+925.72/0.836619.56/0.630215.01/0.3641ISTA-Net+930.49/0.830924.81/0.705720.00/0.5411
本 文28.55/0.883924.51/0.701718.62/0.4116本 文33.02/0.869429.71/0.742124.76/0.5832
FlinstonesReconNet418.92/0.609916.30/0.442013.96/0.2475PeppersReconNet422.15/0.745119.56/0.525816.82/0.4223
DR2-Net521.09/0.672316.93/0.460614.01/0.2499DR2-Net523.73/0.765820.32/0.534216.90/0.4226
ISTA-Net923.39/0.744117.43/0.465114.00/0.2497ISTA-Net926.66/0.799420.64/0.594116.88/0.4227
ISTA-Net+923.39/0.751117.65/0.495513.90/0.2454ISTA-Net+927.13/0.810721.10/0.617716.94/0.4255
本 文23.94/0.767320.78/0.521616.97/0.2696本 文27.58/0.834324.97/0.643921.23/0.417
CameramanReconNet421.28/0.678419.26/0.583617.11/0.4619MeanReconNet422.68/0.685719.99/0.520217.27/0.4092
DR2-Net522.46/0.712519.84/0.593117.08/0.4588DR2-Net524.32/0.718820.80/0.562917.44/0.4108
ISTA-Net923.46/0.749020.27/0.597117.26/0.4643ISTA-Net926.26/0.796121.15/0.596117.36/0.4082
ISTA-Net+923.76/0.753920.50/0.621517.32/0.4763ISTA-Net+926.48/0.803621.56/0.624017.45/0.4131
本 文25.89/0.794723.34/0.651020.77/0.4998本 文28.50/0.845025.52/0.677821.54/0.4536

图5

3种采样率下重建图像的视觉效果"

表2

本文方法与MSRNet,仅多尺度生成网络的PSNR比较"

模型3种采样率的平均PSNR
0.100.040.01
MSRNet625.1621.4117.54
仅多尺度生成网络28.0125.1321.27
本文生成对抗网络28.5025.5221.54

表3

3种尺度下生成网络的PSNR和SSIM对比"

测试集采样率单通道双通道三通道
PSNR/SSIMPSNR/SSIMPSNR/SSIM
Set50.0123.74/0.621924.05/0.639124.15/0.6453
0.0427.89/0.781628.54/0.812728.58/0.8136
0.1031.10/0.870931.64/0.886731.75/0.8892
Set140.0122.47/0.540822.74/0.556822.79/0.5612
0.0425.57/0.675826.09/0.698226.07/0.6989
0.1027.96/0.786028.36/0.798128.47/0.8006

表4

生成网络中有无注意力机制模块的PSNR对比"

模型3种采样率的平均PSNR
0.100.040.01
无注意力机制26.4323.9620.22
有注意力机制28.0125.1321.27
1 Donoho D. Compressed sensing[J]. IEEE Transactions on Information Theory, 2006, 52(4): 1289-1306.
2 Shi W, Jiang F, Liu S, et al. Image compressed sensing using convolutional neural network[J]. IEEE Transactions on Image Processing, 2020, 29: 375-388.
3 Mousavi A, Patel A B, Baraniuk R G. A deep learning approach to structured signal recovery[C]∥The 53rd Annual Allerton Conference on Communication, Control and Computing, Monticello,USA, 2015: 1336-1343.
4 Kulkarni K, Lohit S, Turaga P, et al. ReconNet: non-iterative reconstruction of images from compressively sensed measurements[C]∥IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, USA, 2016: 117-124.
5 Yao H T, Dai F, Zheng D M. DR2-Net:deep residual reconstruction network for image compressive sensing[C]∥IEEE Conference on Computer Vision and Pattern Recognition, Honolulu,USA, 2017: 451-462.
6 练秋生, 富利鹏, 陈书贞, 等. 基于多尺度残差网络的压缩感知重构算法[J]. 自动化学报, 2019, 45(11):10-19.
Lian Qiu-sheng, Fu Li-peng, Chen Shu-zhen, et al. A compressed sensing algorithm based on multi-scale residual reconstruction network[J]. Acta Automatica Sinica, 2019, 45(11): 10-19.
7 Gunning D, Aha D W. Darpa's explainable artificial intelligence program[J]. Magazine, 2019, 40(2): 44-58.
8 Monga V, Li Y, Eldar Y C. Algorithm unrolling: interpretable, efficient deep learning for signal and image processing[J]. IEEE Signal Processing Magazine, 2021, 38(2): 18-44.
9 Zhang J. ISTANet: interpretable optimization-inspired deep network for image compressive sensing[C]∥IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, USA, 2018: 1828-1837.
10 Fergus R, Taylor G W. Adaptive deconvolutional networks for mid and high level feature learning[C]∥International Conference on Computer Vision of IEEE Computer Society, Barcelona,Spain, 2011: 2018-2025.
11 Zeng K, Wang Y, Mao J. Deep residual deconvolutional networks for defocus blur detection[J]. IET Image Processing, 2021, 15: 1-3.
12 Woo S, Park J, Lee J Y, et al. CBAM: convolutional block attention module[C]∥Proceedings of the European Conference on Computer Vision, Munich, Germany, 2018: 3-19..
13 Hou Q, Zhou D, Feng J. Coordinate attention for efficient mobile network design[C]∥Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Online, 2021: 13713-13722.
14 Hu A, Chen S, Wu L. WSGAN: an improved generative adversarial network for remote sensing image road network extraction by weakly supervised processing[J]. Remote Sensing, 2021, 13(13): 2506-2516.
15 Cui T, Gou G, Xiong G.6GAN: IPv6 multi-pattern target generation via generative adversarial nets with reinforcement learning[C]∥IEEE Conference on Computer Communications, Beijing, China, 2021: 1-10.
16 LEDIG C. Photo-realistic single image super-resolution using a generative adversarial network[C]∥IEEE Conference on Computer Vision and Pattern Recognition, Honolulu,USA, 2017: 105-114.
17 Deng Z, Zhu L, Hu X. Deep multi-model fusion for single-image dehazing[C]∥International Conference on Computer Vision, Seoul, South Korea, 2019: 165-174.
[1] 霍光,林大为,刘元宁,朱晓冬,袁梦,盖迪. 基于多尺度特征和注意力机制的轻量级虹膜分割模型[J]. 吉林大学学报(工学版), 2023, 53(9): 2591-2600.
[2] 何颖,王卓然,周旭,刘衍珩. 融合社交地理信息加权矩阵分解的兴趣点推荐算法[J]. 吉林大学学报(工学版), 2023, 53(9): 2632-2639.
[3] 张云佐,董旭,蔡昭权. 拟合下肢几何特征的多视角步态周期检测[J]. 吉林大学学报(工学版), 2023, 53(9): 2611-2619.
[4] 肖明尧,李雄飞,朱芮. 基于NSST域像素相关分析的医学图像融合[J]. 吉林大学学报(工学版), 2023, 53(9): 2640-2648.
[5] 赵亚慧,李飞雨,崔荣一,金国哲,张振国,李德,金小峰. 基于跨语言预训练模型的朝汉翻译质量评估[J]. 吉林大学学报(工学版), 2023, 53(8): 2371-2379.
[6] 车翔玖,徐欢,潘明阳,刘全乐. 生物医学命名实体识别的两阶段学习算法[J]. 吉林大学学报(工学版), 2023, 53(8): 2380-2387.
[7] 王连明,吴鑫. 基于姿态估计的物体3D运动参数测量方法[J]. 吉林大学学报(工学版), 2023, 53(7): 2099-2108.
[8] 张则强,梁巍,谢梦柯,郑红斌. 混流双边拆卸线平衡问题的精英差分进化算法[J]. 吉林大学学报(工学版), 2023, 53(5): 1297-1304.
[9] 张振海,季坤,党建武. 基于桥梁裂缝识别模型的桥梁裂缝病害识别方法[J]. 吉林大学学报(工学版), 2023, 53(5): 1418-1426.
[10] 刘培勇,董洁,谢罗峰,朱杨洋,殷国富. 基于多支路卷积神经网络的磁瓦表面缺陷检测算法[J]. 吉林大学学报(工学版), 2023, 53(5): 1449-1457.
[11] 姜宇,潘家铮,陈何淮,符凌智,齐红. 基于分割方法的繁体中文报纸文本检测[J]. 吉林大学学报(工学版), 2023, 53(4): 1146-1154.
[12] 吕卫,韩镓泽,褚晶辉,井佩光. 基于多模态自注意力网络的视频记忆度预测[J]. 吉林大学学报(工学版), 2023, 53(4): 1211-1219.
[13] 于鹏,朴燕. 基于多尺度特征的行人重识别属性提取新方法[J]. 吉林大学学报(工学版), 2023, 53(4): 1155-1162.
[14] 潘弘洋,刘昭,杨波,孙庚,刘衍珩. 基于新一代通信技术的无人机系统群体智能方法综述[J]. 吉林大学学报(工学版), 2023, 53(3): 629-642.
[15] 何颖,樊俊松,王巍,孙庚,刘衍珩. 无人机空地安全通信与航迹规划的多目标联合优化方法[J]. 吉林大学学报(工学版), 2023, 53(3): 913-922.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!