吉林大学学报(工学版) ›› 2025, Vol. 55 ›› Issue (3): 1015-1027.

• 计算机科学与技术 • 上一篇    

基于自适应大核注意力的轻量级图像超分辨率网络

程德强1(),刘规1,寇旗旗3,张剑英1,江鹤1,2()   

  1. 1.中国矿业大学 信息与控制工程学院,江苏 徐州 221116
    2.成都大学 模式识别与智能信息处理四川省高校重点实验室,成都 610106
    3.中国矿业大学 计算机科学与技术学院,江苏 徐州 221116
  • 收稿日期:2023-10-26 出版日期:2025-03-01 发布日期:2025-05-20
  • 通讯作者: 江鹤 E-mail:cdqcumt@126.com;jianghe@cumt.edu.cn
  • 作者简介:程德强(1979-),男,教授,博士.研究方向:图像智能检测与模式识别,图像处理与视频编码.E-mail:cdqcumt@126.com
  • 基金资助:
    国家自然科学基金项目(52304182);四川省数据恢复重点实验室开放基金项目(DRN2408)

Lightweight image super⁃resolution network based on adaptive large kernel attention fusion

De-qiang CHENG1(),Gui LIU1,Qi-qi KOU3,Jian-ying ZHANG1,He JIANG1,2()   

  1. 1.School of Information and Control Engineering,China University of Mining and Technology,Xuzhou 221116,China
    2.Key Laboratory of Pattern Recognition and Intelligent Information Processing,Institutions of Higher Education of Sichuan Province,Chengdu 610106,China
    3.School of Computer Science and Technology,China University of Mining and Technology,Xuzhou 221116,China
  • Received:2023-10-26 Online:2025-03-01 Published:2025-05-20
  • Contact: He JIANG E-mail:cdqcumt@126.com;jianghe@cumt.edu.cn

摘要:

针对高性能图像超分辨网络通常参数量庞大的问题,提出了一种轻量级模型。首先,集成了3种大核注意力,即双路大核注意力、大核像素注意力和大核通道注意力,旨在扩大模型感受野,建立像素的长程依赖性。其次,引入了自适应注意力融合机制,增强了特征的表征能力,提升了模型性能。实验证明:本文模型在视觉感知和量化测试上表现优异。在Urban100数据集上,与目前流行的ARRFN算法相比,4倍重建结果的峰值信噪比均值提高了0.25 dB。重建图像视觉效果更逼真、纹理更为清晰和自然,充分证明了该算法的有效性。

关键词: 计算机视觉, 超分辨, 大核注意力, 轻量化, 自适应特征融合

Abstract:

To address the issue of large parameter quantities in high-performance image super-resolution networks, a lightweight model was proposed. First, three kinds of large kernel attention were integrated, namely dual-path large kernel attention, large kernel pixel attention and large kernel channel attention, aiming to expand the model perceptual field and establish the long-range dependence of pixels. Second, an adaptive attention fusion mechanism was introduced to enhance the characterization of features and improve the model performance. Experiments demonstrate that the model performs well on visual perception and quantitative tests. On the Urban100 dataset, the mean value of peak signal-to-noise ratio of ×4 reconstruction results was improved by 0.25 dB compared with the currently popular ARRFN algorithm. The reconstructed images have more realistic visual effects, clearer and more natural texture information, which fully demonstrates the proposed algorithm effectiveness.

Key words: computer version, super-resolution, large kernel attention, lightweight, adaptive feature fusion

中图分类号: 

  • TP391.41

图1

自适应大核注意力融合网络框架"

表1

大核注意力(LKA)与多路机制对双路大核注意力(DLKA)性能的影响"

LKA

3-5-1

LKA

5-7-1

LKA

7-9-1

参数量/K浮点运算量/G

Set5

PSNR/SSIM

Set14

PSNR/SSIM

B100

PSNR/SSIM

Urban100

PSNR/SSIM

××767.96113.6937.93/0.958733.43/0.915232.08/0.897531.74/0.9237
××778.20115.2037.94/0.959233.46/0.915932.10/0.897831.74/0.9238
××791.01117.3137.93/0.958433.44/0.915432.09/0.897731.73/0.9234
×837.08123.7337.97/0.959833.50/0.916832.12/0.898331.80/0.9243
×864.25127.3537.98/0.960133.51/0.917132.11/0.898131.78/0.9238
×853.47125.8437.96/0.959433.50/0.916632.10/0.897631.77/0.9235
904.38133.4737.98/0.960433.52/0.917032.11/0.898431.80/0.9246

表2

大核通道注意力(LKCA)的消融实验"

1×1Conv7×7DPConv1×1Conv参数量/M浮点运算量/G

Set5

PSNR/SSIM

Set14

PSNR/SSIM

B100

PSNR/SSIM

Urban100

PSNR/SSIM

×

×

×

××0.5866.5637.76/0.959033.22/0.913531.92/0.896031.12/0.9171
×0.6068.9837.78/0.959533.23/0.913631.93/0.896131.13/0.9173
×0.5968.4137.79/0.959633.24/0.913931.93/0.896231.13/0.9172
×0.6171.3737.77/0.959233.23/0.913631.93/0.896231.12/0.9172
0.6373.2237.81/0.959333.26/0.914231.94/0.896331.14/0.9175

图2

Set5中的woman重建结果"

表3

双路大核注意力模块(DLKA)和大核像素注意力(LKPA)以及自适应权重学习模块(AWLM)对自适应空间大核注意力(ALKA)模块性能的影响"

DLKALKPAAWLM参数量/M浮点运算量/G

Set5

PSNR/SSIM

Set14

PSNR/SSIM

B100

PSNR/SSIM

Urban100

PSNR/SSIM

×

×

××0.74109.9937.81/0.957333.34/0.914232.03/0.896931.62/0.9213
××0.84123.7337.97/0.959833.50/0.916832.12/0.898331.80/0.9243
×0.77113.3737.93/0.958933.47/0.916132.11/0.898031.73/0.9234
×0.86127.1138.02/0.960833.50/0.916932.12/0.898331.98/0.9270
0.88127.1838.04/0.961033.52/0.917132.14/0.898732.01/0.9275

表4

自适应空间大核注意力模块(ALKA)和大核通道注意力(LKCA)对特征提取块(LAFB)性能的影响"

ALKALKCA参数量/M浮点运算量/G

Set5

PSNR/SSIM

Set14

PSNR/SSIM

B100

PSNR/SSIM

Urban100

PSNR/SSIM

×

×

×0.74109.9937.81/0.957333.34/0.914232.03/0.896931.62/0.9213
×0.88127.1838.04/0.960933.52/0.917132.14/0.898732.01/0.9275
0.92116.7337.96/0.959433.43/0.915232.06/0.897231.65/0.9217
1.06133.9038.05/0.961033.56/0.917532.16/0.899132.04/0.9279

表5

双路重建策略对网络模型性能的影响"

双路重建

Set5

PSNR/SSIM

Set14

PSNR/SSIM

B100

PSNR/SSIM

Urban100

PSNR/SSIM

×38.04/0.960933.51/0.916932.15/0.898732.00/0.9269
38.05/0.961033.56/0.917532.16/0.899132.04/0.9279

图3

Urban100中Img011重建结果"

图4

Urban100中Img005重建结果"

图5

Urban100中Img033重建结果"

图6

B100中182053重建结果"

表6

缩放因子分别为2、3、4时在基准数据集上的指标对比"

缩放因子模型参数量/M浮点运算量/G

Set524

PSNR/SSIM

Set1425

PSNR/SSIM

B10026

PSNR/SSIM

Urban10027

PSNR/SSIM

x2SRCNN100.0552.736.66/0.954232.42/0.906331.36/0.887929.50/0.8946
FSRCNN280.016.037.00/0.955832.63/0.908831.53/0.892029.88/0.9020
VDSR290.66612.637.53/0.958733.03/0.912431.90/0.896030.76/0.9140
DRCN301.7717974.037.63/0.958833.04/0.911831.85/0.894230.75/0.9133
CARN311.59222.837.76/0.959033.52/0.916632.09/0.897831.92/0.9256
EDSR-baseline121.37316.237.99/0.960433.57/0.917532.16/0.899431.98/0.9272
IMDN320.69158.838.00/0.960533.63/0.917732.19/0.899632.17/0.9283
LAMRN351.39320.138.09/0.960833.78**/0.9195**32.22/0.900332.31/0.9299
FMEN340.75172.038.10*/0.9609*33.75*/0.9192*32.26**/0.9007**32.41/0.9311
A2F-L331.36306.138.09/0.960733.78**/0.9192*32.23/0.900232.46*/0.9313*
ARRFN360.98-38.01/0.960633.66/0.917932.20/0.899932.27/0.9295
ALSR(本文)1.51189.338.14**/0.9616**33.74/0.918732.24*/0.9004*32.48**/0.9317**
x3SRCNN100.0552.732.75/0.909029.28/0.820928.41/0.786326.24/0.7989
FSRCNN280.015.033.16/0.914029.43/0.824228.53/0.791026.43/0.8080
VDSR290.66612.633.66/0.921329.77/0.831428.82/0.797627.14/0.8279
DRCN301.7717974.033.82/0.922629.76/0.831128.80/0.796327.15/0.8276
CARN311.59118.834.29/0.925530.29/0.840729.06/0.803428.06/0.8493
EDSR-baseline121.55160.434.37/0.927030.28/0.841729.09/0.805228.15/0.8527
IMDN320.7071.534.36/0.927030.32/0.841729.09/0.804628.17/0.8519
LAMRN351.41145.334.55*/0.9285*30.41*/0.843129.17*/0.8067*28.43*/0.8571
FMEN340.7677.234.45/0.927530.40/0.843529.17*/0.806328.33/0.8562
A2F-L331.37136.334.54/0.928330.41*/0.8436*29.14/0.806228.40/0.8574*
ARRFN360.99-34.38/0.927230.36/0.842229.09/0.805028.22/0.8533
ALSR(本文)1.69122.534.58**/0.9293**30.47**/0.8451**29.18**/0.8069**28.44**/0.8576**
x4SRCNN100.0552.730.48/0.862827.49/0.750326.90/0.710124.52/0.7221
FSRCNN280.014.630.71/0.865727.59/0.753526.98/0.715024.62/0.7280
VDSR290.66612.631.35/0.883828.01/0.767427.29/0.725125.18/0.7524
DRCN301.7717974.031.53/0.885428.02/0.767027.23/0.723325.14 /0.7510
CARN311.5990.932.13/0.893728.60/0.780627.58/0.734926.07/0.7837
EDSR-baseline121.51114.232.09/0.893828.58/0.781327.57/0.735726.04/0.7849
IMDN320.7040.932.21/0.894828.58/0.781127.56/0.735326.04/0.7838
LAMRN351.4185.032.37*/0.8971*28.70*/0.7839*27.61/0.7384*26.32*/0.7925
FMEN340.7744.232.24/0.895528.70*/0.7839*27.63/0.737926.28/0.7908
A2F-L331.3777.232.32/0.896428.67/0.783927.62*/0.737926.32*/0.7931*
ARRFN361.00-32.22/0.895228.60/0.781727.57/0.735526.09/0.7858
ALSR(本文)1.6669.932.40**/0.8976**28.71**/0.7843**27.65**/0.7386**26.34**/0.7937**

表7

与基于Transformer的算法对比"

尺度因子LBNet37ESRT18NGswin38ALSR(本文)
2-38.03/0.960038.05/0.961038.14/0.9616
334.47/0.927734.42/0.926834.52/0.928234.58/0.9293
432.29/0.896032.19/0.894732.33/0.896332.40/0.8976

图7

Urban100中Img092的差异图"

图8

B100中351093的差异图"

表8

模型复杂度对模型性能的影响"

LAFB个数输出通道数参数量/M浮点运算量/G

Set5

PSNR/SSIM

Set14

PSNR/SSIM

B100

PSNR/SSIM

Urban100

PSNR/SSIM

4641.06133.9038.09/0.960933.64/0.916432.19/0.899532.17/0.9286
4962.33295.0738.15/0.961933.72/0.918432.26/0.900932.45/0.9311
6641.51189.0338.14/0.961633.74/0.918732.24/0.900432.48/0.9317
6963.33417.7838.18/0.962133.83/0.919732.29/0.901132.60/0.9327
1 Cheng D Q, Chen L L, Lv C, et al. Light-guided and cross-fusion U-Net for anti-illumination image super-resolution[J]. IEEE Transaction on Circuits and Systems for Video Technology, 2022, 32(12): 8436-8449.
2 Jebkins W K, Mather B C, Jr D C M. Nearest neighbor and generalized inverse distance interpolation for fourier domain image reconstruction[C]∥Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), Tampa, USA, 1985:1069-1072.
3 Blu T, Thevenaz P, Unser M. Linear interpolation revitalized[J]. IEEE Trans Image Process, 2004, 13(5): 710-719.
4 Lin C, Shen M, Chang H, et al. The efficient VLSI design of BICUBIC convolution interpolation for digital image processing[C]∥Proceedings of the International Symposium on Circuitsand Systems (ISCAS), Seattle, USA, 2008:480-483.
5 Dai S, Han M, Xu W,et al. Soft-cuts: a soft edge smoothness prior for color image super-resolution[J]. IEEE Transactions on Image Processing, 2009, 18(5): 969-981.
6 Sun J, Xu Z, Shun H Y. Image super-resolution using gradient profile prior[C]∥Proceedings of the IEEE Conference on Computer Vision and Pattern Reco Anchorage, Anchorage, AK, USA, 2008: 1-8.
7 Yan Q, Xu Y, Yang X, et al. Single image super resolution based on gradient profile sharpness[J]. IEEE Transactions on Image Processing, 2015,24(10):3187-3202 .
8 iang H J, Mujtaba A S A D, Liu J J, et al. Single image detail enhancement via metropolis theorem[J]. Multimedia Tools and Applications,2024, 83: 36329-36353.
9 程德强, 郭昕, 陈亮亮, 等. 多通道递归残差网络的图像超分辨率重建[J].中国图象图形学报, 2021, 26(3): 605-618.
Cheng De-qiang, Guo Xin, Chen Liang-liang, et al. Image super resolution reconstruction by multi-channel recursive residual networks[J]. Journal of Image and Graphics, 2021, 26(3): 605-618.
10 Dong C, Loy C C, He K M, et al. Image super-resolution using deep convolutional networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2016, 38(2): 295-307.
11 Shi W, Caballero J, Huszár F,et al. Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network[C]∥IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, USA, 2016:1874-1883.
12 Lim B, Son S, Kim H,et al. Enhanced deep residual networks for single image super-resolution[C]∥IEEE Conference on Computer Vision and Pattern Recognition Workshops, Honolulu,USA, 2016:1132-1140.
13 郭继昌, 吴洁, 郭春乐, 等. 基于残差连接卷积神经网络的图像超分辨率重构[J]. 吉林大学学报: 工学版,2019, 49(5): 1726-1734.
Guo Ji-chang, Wu Jie, Guo Chun-le, et al.Image super-resolution reconstruction based on residual-connected convolutional neural networks[J]. Journal of Jilin University(Engineering and Technology Edition,2019, 49(5): 1726-1734.
14 Zhang Y, Li K. Image super-resolution using very deep residual channel attention networks[C]∥Proceedings of the European Conference on Computer Vision, Munich Germany, 2018: 286-301.
15 Niu B, Wen W L, Ren W Q,et al. Single image super-resolution via a holistic attention network[C]∥ 2020 European Conference on Computer Vision (ECCV), Glasgow, USA, 2020:191-207.
16 Vaswan A, Shazeer N, Parmar, et al. Attention is all you need[J/OL].[2023-09-15].
17 Liang J Y, Cao J Z, Sun G L,et al. Image restoration using swin transformer[C]∥Proceedings of the IEEE/CVF International Conference on Computer Vision, Montreal, Canada, 2021: 1833-1844.
18 Lu Z S, Li J C, Liu H, et al. Transformer for single image super-resolution[C]∥Proc IEEE/CVF Conf Comput Vis Pattern Recognit Workshops, New Orleans, USA, 2022: 456-465.
19 Kou Q Q, Cheng D Q, Zhang H et al. Single Image super resolution via multi-attention fusion recurrent network[J]. IEEE Access,2023(11): 98653-98665.
20 Wang Z, Gao G, Li J,et al. Lightweight image super-resolution with multi-scale feature interaction network[C]∥IEEE International Conference on Multimedia and Expo, Shenzhen, China, 2021: No.9428136.
21 Lan R, Sun L, Liu Z,et al. MADNet:a fast and lightweight network for single-image super resolution[J]. IEEE Transactions on Cybernetics, 2021,51(3):1443-1453.
22 Guo M H, Lu C Z, Liu Z N,et al. Visual attention network[J/OL].[2023-09-18].
23 Jie H, Li S, Samuel A, et al. Squeeze-and-excita⁃tion networks[C]∥Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recogni⁃tion, Wellington, New Zealand, 2018: 7132-7141.
24 Bevilacqua M, Roumy A, Guillemot C, et al. Low-complexity single image super-resolution based on nonnegative neighbour embedding[C]∥British Machine Vision Conference, Oxford,UK, 2012:1-10.
25 Zeyde R, Elad M, Protter M. On single image scale-up using sparse-representations[C]∥International Conference on Curves and Surfaces,Avignon, France, 2010: 711-730.
26 Kim J, Lee J K, Lee K M. Accurate image super-resolution using very deep convolutional networks[C]∥2016 IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, USA, 2016:1646-1654.
27 Huang J B, Singh A, Ahuja N. Single image super-resolution from transformed self-exemplars[C]∥IEEE Conference on Computer Vision and Pattern Recognition,Boston, USA, 2015:5197-5206.
28 Dong C, Loy C C, Tang X. Accelerating the super-resolution convolutional neural network[C]∥European Conference on Computer Vision,Amsterdam, Holland, 2016:391-407.
29 Kim J, Lee J K, Lee K M. Accurate image super-resolution using very deep convolutional networks[C]∥Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, USA, 2016: 1646-1654.
30 Kim J, Lee J K, Lee K M. Deeply-recursive convolutional network for image super-resolution[C]∥Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, USA,2016:1637-1645.
31 Ahh N, Kang B, Sohn K A. Fast, accurate, and lightweight super-resolution with cascading residual network[C]∥European Conference on Computer Vision(ECCV), Munich, Germany, 2018: 256-272.
32 Hui Z, Gao X, Yang Y, et al. Lightweight image super-resolution with information multi-distillation network[C]∥Proceedings of the 27th ACM International Conference on Multimedia, Xi'an, China, 2019: 2024-2032.
33 Wang X H, Wang Q, Zhao Y Z, et al. Lightweight single-image super-resolution network with attentive auxiliary feature learning[C]∥Asian Conference on Computer Vision(ACCV),Kyoto, Japan, 2020: 3023-3041.
34 Du Z C, Liu D, Liu J, et al. Fast and memory-efficient network towards efficient image super-resolution[C]∥Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition,New Orleans, USA, 2022 :853-862.
35 Yan Y, Xu X, Chen W, et al. Lightweight attended multi-scale residual network for single image super-resolution[J]. IEEE Access, 2021, 9: 52202-52212.
36 Qin J, Zhang R. Lightweight single image super-resolution with attentive residual refinement network[J].Neurocomputing, 2022, 500: 846-855.
37 Gao G W, Wang Z X, Li J C, et al. Lightweight bimodal network for single-image super-resolution via symmetric CNN and recursive transformer[C]∥International Joint Conference on Artificial Intelligence,Shenzhen, China, 2022: 913-919.
38 Choi H, Lee J, Yang J. N-Gram in swin transformers for efficient lightweight image super-resolution[C]∥Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Vancouver, Canadian, 2023: 2071-2081.
39 Jie H, Li S, Samuel A, et al. Squeeze-and-excitation networks[C]∥Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Wellington, New Zealand, 2018: 7132-7141.
[1] 董华松,连远锋. 海量数字媒体视频无损转码重压缩的轻量化检测算法[J]. 吉林大学学报(工学版), 2025, 55(2): 741-747.
[2] 才华,郑延阳,付强,王晟宇,王伟刚,马智勇. 基于多尺度候选融合与优化的三维目标检测算法[J]. 吉林大学学报(工学版), 2025, 55(2): 709-721.
[3] 张曦,库少平. 基于生成对抗网络的人脸超分辨率重建方法[J]. 吉林大学学报(工学版), 2025, 55(1): 333-338.
[4] 朱圣杰,王宣,徐芳,彭佳琦,王远超. 机载广域遥感图像的尺度归一化目标检测方法[J]. 吉林大学学报(工学版), 2024, 54(8): 2329-2337.
[5] 才华,寇婷婷,杨依宁,马智勇,王伟刚,孙俊喜. 基于轨迹优化的三维车辆多目标跟踪[J]. 吉林大学学报(工学版), 2024, 54(8): 2338-2347.
[6] 郭昕刚,何颖晨,程超. 抗噪声的分步式图像超分辨率重构算法[J]. 吉林大学学报(工学版), 2024, 54(7): 2063-2071.
[7] 孙铭会,薛浩,金玉波,曲卫东,秦贵和. 联合时空注意力的视频显著性预测[J]. 吉林大学学报(工学版), 2024, 54(6): 1767-1776.
[8] 王殿伟,张池,房杰,许志杰. 基于高分辨率孪生网络的无人机目标跟踪算法[J]. 吉林大学学报(工学版), 2024, 54(5): 1426-1434.
[9] 王宇,赵凯. 基于亚像素定位的人体姿态热图后处理[J]. 吉林大学学报(工学版), 2024, 54(5): 1385-1392.
[10] 张自超,陈建. 基于双目仿鹰眼视觉与超分辨的果园三维点云重建[J]. 吉林大学学报(工学版), 2024, 54(5): 1469-1481.
[11] 高云龙,任明,吴川,高文. 基于注意力机制改进的无锚框舰船检测模型[J]. 吉林大学学报(工学版), 2024, 54(5): 1407-1416.
[12] 毛琳,苏宏阳,杨大伟. 时间显著注意力孪生跟踪网络[J]. 吉林大学学报(工学版), 2024, 54(11): 3327-3337.
[13] 孙文财,胡旭歌,杨志发,孟繁雨,孙微. 融合GPNet与图像多尺度特性的红外-可见光道路目标检测优化方法[J]. 吉林大学学报(工学版), 2024, 54(10): 2799-2806.
[14] 李本怀,刘艳文,王璐,陈雪乾,左文杰. 吸能与压溃位移约束下的轨道车辆前端结构抗撞性优化[J]. 吉林大学学报(工学版), 2023, 53(4): 982-988.
[15] 刘晶红,邓安平,陈琪琪,彭佳琦,左羽佳. 基于多重注意力机制的无锚框目标跟踪算法[J]. 吉林大学学报(工学版), 2023, 53(12): 3518-3528.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!