融合对比学习和生成对抗网络的图像去雾算法

doi:10.13229/j.cnki.jdxbgxb.20240111

摘要/Abstract

摘要：

针对目前去雾算法依赖有雾、无雾图像对的局限，以及监督学习导致的成本消耗等问题，提出了一种基于对比学习和循环一致性生成对抗网络的图像去雾算法。首先，通过非成对的有雾图像和清晰图像训练循环一致性生成对抗网络，提高图像去雾算法在真实场景中的应用价值，缓解去雾算法的域偏移问题；其次，设计对比指导分支学习图像的潜在特征分布，隐式约束不同样本在深度特征空间中的嵌入信息，深入挖掘有雾图像和清晰图像的相似特征，拉近图像相似特征的距离，保留两类图像间的互信息，维持图像内容的一致性，提高网络去雾性能；然后，引入频率损失函数，约束生成器的输出，降低频域信息损失，进一步保留图像的内容和结构信息，减少去雾图像的模糊和失真，提高生成图像的质量和清晰度。实验结果表明，本文模型相比目前主流的基于深度学习的传统去雾算法，信息熵和平均梯度均有所提高，细节信息更丰富，是一种有效的图像去雾算法。

关键词: 图像去雾, 非成对图像, 生成对抗网络, 对比学习

Abstract:

Aiming at the limitations of some current defogging algorithms caused by using foggy and non-foggy image pairs and the cost consumption caused by supervised learning， this paper proposes an image defogging algorithm based on comparative learning and recurrent consistent generative adversarial network. By training recurrent generative adversarial network with unpaired foggy and clear images， the value of image defogging algorithm in real scenes is improved， and the domain shift problem of defogging algorithm is alleviated； meanwhile， we design the contrast-guided branch to learn the potential feature distribution of the image， implicitly constrain the embedding of different samples in the depth feature space， deeply mine the similar features of foggy and clear images， pull the similar characteristics of the images closer together， retain the mutual information between the two types of images， maintain the consistency of image content， and improve the performance of network defogging； introduce the frequency loss， constrain the output of the generator， reduce the loss of information in the frequency domain， further retain the content and structural information of the image， reduce the blurring and distortion of the defogged image， and improve the quality and clarity of the generated image. Experimental results show that the model proposed in this paper is an effective image defogging algorithm with improved information entropy and average gradient and richer detail information compared to the current mainstream deep learning-based and traditional defogging algorithms.

Key words: image defogging, unpaired images, generative adversarial networks, contrast learning

中图分类号:

TP391

罗向龙,魏欣语,赵茂军,刘若辰. 融合对比学习和生成对抗网络的图像去雾算法[J]. 吉林大学学报(工学版), 2025, 55(10): 3296-3308.

Xiang-long LUO,Xin-yu WEI,Mao-jun ZHAO,Ruo-chen LIU. Image dehazing algorithm based on contrast learning and generative adversarial network[J]. Journal of Jilin University(Engineering and Technology Edition), 2025, 55(10): 3296-3308.

图/表 20

图1

图2

图3

图4

图5

图6

图7

图8

图9

图10

表1

生成器G详细参数"

生成器 $G$	卷积核大小 $k$ 与数量 $n$	网络层输出
编码模块	$k = 7, n = 64$	$256 × 256 × 64$
	$k = 3, n = 128$	$128 × 128 × 128$
	$k = 3, n = 256$	$64 × 64 × 256$
多尺度特征提取模块（ $× 3$ ）	$k = 1, n = 64$	$64 × 64 × 256$
	$k = 1, n = 128; k = 3, n = 64$
	$k = 1, n = 128; k = 5, n = 64$
	最大池化， $k = 1, n = 64$
	$k = 3, n = 256, k = 3, n = 256$	$64 × 64 × 256$
	$k = 3, n = 256, k = 3, n = 256$	$64 × 64 × 256$
解码模块	$k = 3, n = 128$	$128 × 128 × 128$
	$k = 3, n = 64$	$256 × 256 × 64$
	$k = 7, n = 3$	$256 × 256 × 3$

表1

表2

生成器F详细参数"

生成器 $F$	网络层参数	网络层输出
编码模块	$k = 3, n = 64, s = 1$ $k = 3, n = 64, s = 1$ $k = 3, n = 64, s = 2$ $k = 3, n = 128, s = 1$ $k = 3, n = 128, s = 1$ $k = 3, n = 128, s = 2$ $k = 3, n = 256, s = 1$ $k = 3, n = 256, s = 1$ $k = 3, n = 256, s = 2$ $k = 3, n = 512, s = 1$ $k = 3, n = 512, s = 1$ $k = 3, n = 512, s = 2$	$256 × 256 × 64$ $128 × 128 × 64$ $128 × 128 × 128$ $64 × 64 × 128$ $64 × 64 × 256$ $32 × 32 × 256$ $32 × 32 × 512$ $16 × 16 × 512$
解码模块	$k = 3, n = 1024, s = 1$ $k = 3, n = 1024, s = 1$	$16 × 16 × 1024$
	$k = 3, n = 512, s = 2, o p = 1$	$32 × 32 × 512$
	$k = 3, n = 512, s = 1$ $k = 3, n = 512, s = 1$	$32 × 32 × 512$
	$k = 3, n = 256, s = 2, o p = 1$	$64 × 64 × 256$
	$k = 3, n = 256, s = 1$ $k = 3, n = 256, s = 1$	$64 × 64 × 256$
	$k = 3, n = 128, s = 2, o p = 1$	$128 × 128 × 128$
	$k = 3, n = 128, s = 1$ $k = 3, n = 128, s = 1$	$128 × 128 × 128$
	$k = 3, n = 64, s = 2, o p = 1$	$256 × 256 × 64$
	$k = 3, n = 64, s = 1$ $k = 3, n = 64, s = 1$	$256 × 256 × 64$
	$k = 3, n = 3, s = 1$	$256 × 256 × 3$

表2

表3

判别器详细参数"

判别器模块	卷积核大小 $k$ 与数量 $n$	网络层输出
第一层	$k = 3, n = 64$	$128 × 128 × 64$
第二层	$k = 3, n = 128$	$64 × 64 × 128$
第三层	$k = 3, n = 256$	$32 × 32 × 256$
第四层	$k = 3, n = 512$	$32 × 32 × 512$
输出	$k = 3, n = 1$	$32 × 32 × 1$

表3

图11

表4

RealData数据集不同算法性能对比"

去雾方法	IE $↑$	AG $↑$	NIQE $↓$
DCP^［6］	7.061	5.533	7.407
CAP^［7］	7.136	5.154	7.800
FFA-Net^［27］	7.033	6.665	7.573
GCA-Net^［28］	7.216	6.145	7.223
CycleGAN^［21］	7.238	7.677	5.951
Cycle-Dehaze^［15］	7.280	7.397	5.338
C-CycleGAN	7.300	8.039	5.464

表4

图12

表5

RTTS数据集不同算法性能对比"

去雾方法	IE $↑$	AG $↑$	NIQE $↓$
DCP^［6］	6.959	5.183	7.040
CAP^［7］	7.065	4.692	7.651
FFA-Net^［27］	7.007	4.681	7.322
GCA-Net^［28］	7.282	6.162	7.761
CycleGAN^［21］	7.225	7.752	5.632
Cycle-Dehaze^［15］	7.274	7.480	4.956
C-CycleGAN	7.292	8.065	5.150

表5

表6

RealData数据集消融实验结果"

实验	IE $↑$	AG $↑$	NIQE $↓$
实验1	7.214	7.458	5.735
实验2	7.269	7.792	5.682
实验3	7.258	7.817	5.896
实验4	7.300	8.039	5.464

表6

图13

表7

RTTS数据集消融实验结果"

实验	IE $↑$	AG $↑$	NIQE $↓$
实验1	7.195	7.475	5.532
实验2	7.284	7.895	5.376
实验3	7.268	7.783	5.521
实验4	7.292	8.065	5.150

表7

参考文献 28

[1]	Bansode N V, Ingale V, Ingale S. An appropriate histogram equalization technique for thermal image enhancement of neck region[C]∥IEEE Pune Section International Conference(PuneCon), Pune, India: IEEE,2023:1-5.
[2]	李英, 李欣玥, 王佳琦, 等. 基于Retinex去雾算法的水射流辅助激光加工特征图像融合算法[J]. 中国激光, 2023, 50(24): 67-76.
	Li Ying, Li Xin-yue, Wang Jia-qi,et al. Water jet assisted laser processing feature image fusion algorithm based on Retinex defogging algorithm[J]. Chinese Journal of Lasers, 2023, 50(24): 67-76.
[3]	Dai W, Ren X M. Defogging algorithm for road environment landscape visual image based on wavelet transform[C]∥International Conference on Networking, Informatics and Computing(ICNETIC), Palermo, Italy, 2023: 587-591.
[4]	Cantor A E J. Optics of the atmosphere: scattering by molecules and particles[J]. IEEE Journal of Quantum Electronics, 1978, 14(9): 698-699.
[5]	Nayar S K, Narasimhan S G. Vision in bad weather [C]∥Proceedings of the Seventh IEEE International Conference on Computer Vision, Kerkyra, Greece, 1999: 820-827.
[6]	He K M, Sun J, Tang X O. Single image haze removal using dark channel prior[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2011, 33(12): 2341-2353.
[7]	Tang J L, Zhang Z D, Niu L H, et al. Research on image defogging algorithms based on color attenuation prior[J]. Journal of Physics: Conference Series, 2022, 2216(1): No. 012080.
[8]	牛宏侠, 王春智, 梁乐观, 等. 基于改进暗通道先验的沙尘图像清晰化算法[J]. 吉林大学学报: 理学版,2023, 61(6): 1407-1418.
	Niu Hong-xia, Wang Chun-zhi, Liang Le-guan, et al. Sand and dust image clarification algorithm based on improved dark channel prior[J]. Journal of Jilin University(Science Edition), 2023, 61(6): 1407-1418.
[9]	许懿娜, 王义, 黄华平, 等. 一种改进暗通道先验的航空影像快速去雾方法[J]. 遥感信息, 2023, 38(6):36-41.
	Xu Yi-na, Wang Yi, Huang Hua-ping, et al. A fast defogging method for aerial images with improved dark channel prior[J]. Remote Sensing Information, 2023, 38(6):36-41.
[10]	王勇, 边宇霄, 李新潮, 等. 基于多尺度编码-解码神经网络的图像去雾算法[J]. 吉林大学学报:工学版, 2024, 54(12):3626-3636.
	Wang Yong, Bian Yu-xiao, Li Xin-chao, et al. Image defogging algorithm based on multi-scale encoding-decoding neural network[J]. Journal of Jilin University(Engineering and Technology Edition), 2024, 54(12):3626-3636.
[11]	李永福, 崔恒奇, 朱浩, 等. 一种基于改进AOD-Net的航拍图像去雾算法[J]. 自动化学报, 2022, 48(6): 1543-1559.
	Li Yong-fu, Cui Heng-qi, Zhu Hao, et al. An improved AOD-Net based defogging algorithm for aerial images[J]. Acta Automatica Sinica, 2022, 48(6): 1543-1559.
[12]	Goodfellow I, Pouget-Abadie J, Mirza M, et al. Generative adversarial networks[J]. Communications of the ACM, 2020, 63(11): 139-144.
[13]	屠杭垚, 王万良, 陈嘉诚, 等. 基于条件生成对抗网络的图像翻译综述[J]. 计算机辅助设计与图形学学报, 2024, 36(1): 14-32.
	Tu Hang-yao, Wang Wan-liang, Chen Jia-cheng, et al. A review of image translation based on conditional generative adversarial networks[J]. Journal of Computer Aided Design and Graphics, 2024, 36(1): 14-32.
[14]	Wang P Y, Zhu H Q, Huang H, et al. TMS-GAN: A twofold multi-scale generative adversarial network for single image dehazing[C]∥IEEE Transactions on Circuits and Systems for Video Technology. IEEE, 2022, 32(5): 2760-2772.
[15]	Engin D, Genç A, Kemal Ekenel H. Cycle-dehaze: Enhanced CycleGAN for single image dehazing[C]∥ IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, Salt Lake City, USA,2018: 825-833.
[16]	Zhang X J, Hu Q, Wei Y N, et al. CycleGAN image defogging method based on residual dual attention mechanism[C]∥Electronic Information Engineering and Computer Science(EIECS), Changchun, China: 2023: 234-237.
[17]	Zhong M Q, Wang X, Wang J, et al. A remote sensing image defogging method based on improved CycleGAN network[C]∥International Conference on Computer Vision, Image and Deep Learning, Zhuhai,China, 2023: 113-116.
[18]	但志平, 方帅领, 孙航, 等. 基于双判别器异构CycleGAN框架下多阶通道注意力校准的室外图像去雾[J]. 电子学报, 2023, 51(9): 2558-2571.
	Dan Zhi-ping, Fang Shuai-ling, Sun Hang, et al. Outdoor image defogging based on dual discriminator heterogeneous CycleGAN framework with multi-order channel attention calibration[J]. Acta Electronica Sinica, 2023, 51(9): 2558-2571.
[19]	钱旭淼, 段锦, 刘举, 等. 基于注意力特征融合的图像去雾算法[J]. 吉林大学学报:理学版, 2023, 61(3): 567-576.
	Qian Xu-miao, Duan Jin, Liu Ju, et al. Image defogging algorithm based on attention feature fusion[J]. Journal of Jilin University(Science Edition), 2023, 61(3):567-576.
[20]	Li B Y, Ren W Q, Fu D P, et al. Benchmarking single-image dehazing and beyond[J]. IEEE Transactions on Image Processing, 2019, 28(1): 492-505.
[21]	Zhu J Y, Park T, Isola P, et al. Unpaired image-to-image translation using cycle-consistent adversarial networks[C]∥IEEE International Conference on Computer Vision(ICCV), Venice, Italy,2017: 2223-2232.
[22]	Shao Y J, Li L, Ren W Q, et al. Domain adaptation for image dehazing [C]∥IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, USA, 2020: 2808-2817.
[23]	Wu H Y, Qu Y Y, Lin S H, et al. Contrastive learning for compact single image dehazing[C]∥IEEE/CVF Conference on Computer Vision and Pattern Recognition, Nashville, USA,2021: 10551-10560.
[24]	Chen X, Pan J S, Jiang K, et al. Unpaired deep image deraining using dual contrastive learning[C]∥IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR), New Orleans, USA, 2022: 2017-2026.
[25]	Isola P, Zhu J Y, Zhou T H, et al. Image-to-image translation with conditional adversarial networks[C]∥IEEE International Women in Engineering(WIE) Conference on Electrical and Computer Engineering, Honolulu,USA,2017: 1125-1134.
[26]	Jiang L M, Dai B, Wu W, et al. Focal frequency loss for image reconstruction and synthesis[C]∥IEEE/CVF International Conference on Computer Vision, Montreal, Canada, 2021: 13899-13909.
[27]	Qin X, Wang Z L, Bai Y C, et al. FFA-Net: Feature fusion attention network for single image dehazing [C]∥AAAI Conference on Artificial Intelligence, New York, USA: 2020: 11908-11915.
[28]	Das S, Saiful Islam M, Ruhul Amin M. GCA-Net: Utilizing gated context attention for improving image forgery localization and detection[C]∥IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops(CVPRW), New Orleans,USA,2022:81-90.

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed