吉林大学学报(工学版) ›› 2023, Vol. 53 ›› Issue (1): 255-262.doi: 10.13229/j.cnki.jdxbgxb20210542

• 通信与控制工程 • 上一篇    

采用VGG19和低通滤波的红外与微光图像融合方法

刘洲洲1(),孙传新2,王晓柱3,张杨梅4   

  1. 1.西安航空学院 计算机学院，西安 710077
    2.中国电子科技集团公司第二十九研究所，成都 610036
    3.香港中文大学（深圳） 数据科学学院，广东 深圳 518172
    4.西安航空学院 电子工程学院，西安 710077
  • 收稿日期:2021-06-18 出版日期:2023-01-01 发布日期:2023-07-23
  • 作者简介:刘洲洲（1981-），男，教授，博士.研究方向：传感网与信号处理.E-mail：nazi2005@126.com
  • 基金资助:
    陕西省教育厅科研计划项目(20JG014);陕西省重点研发计划项目(2020GY-084)

Infrared and low⁃light image fusion based on VGG19 and low⁃pass filtering

Zhou-zhou LIU1(),Chuan-xin SUN2,Xiao-zhu WANG3,Yang-mei ZHANG4   

  1. 1.School of Computer Science，Xi'an Aeronautical University，Xi'an 710077，China
    2.No. 29 Research Institute of CETC，Chengdu 610036，China
    3.School of Data Sciences，The Chinese University of Hong Kong（Shenzhen），Shenzhen 518172，China
    4.School of Electronic Engineering，Xi'an Aeronautical University，Xi'an 710077，China
  • Received:2021-06-18 Online:2023-01-01 Published:2023-07-23

RICH HTML

 2   

PDF (PC)

143

摘要：

针对当前单一传感器受其成像性能的影响，通常很难全面反映地物场景下的全部有效信息，从而产生场景信息难以准确识别的问题，提出了一种采用VGG19与低通滤波的红外与微光图像融合算法。首先，通过红外与微光探测器得到地物场景信息，采用三维分布、直方图对比以及取反方式对图像进行处理，同时分析红外与微光图像目标特性，研究双频图像的光谱机理；其次，在此基础上，利用低通滤波方式分解红外与微光图像，得到其轮廓信息与显著信息，轮廓部分采用平均加权策略进行融合，显著部分采用VGG策略进行多层融合，进而融合重构图像；最后，与其他算法结果进行对比，并利用性能评估方法评价各融合算法。实验结果表明，该算法能够增强图像中场景信息的灰度，可以很好地提高场景亮度，解决了单频图像中场景信息抗背景干扰的问题。

关键词: 红外图像, 微光图像, 低通滤波, VGG网络, 融合算法

Abstract:

Due to the current single sensor being affected by its imaging performance， it is often difficult to fully reflect all the effective information in the ground object scene， resulting in the problem that the scene information is difficult to accurately identify， so a new infrared and low light image fusion algorithm based on VGG and low-pass filtering is proposed. Firstly， obtain the ground object scene information through infrared and low-level light detectors， use three-dimensional distribution， histogram comparison， and inversion to process the image， analyze the target characteristics of infrared and low-level light images， and study the spectroscopic mechanism of dual-frequency images； On this basis， the low-pass filtering method is used to decompose the infrared and low-light images to obtain their contour information and salient information. The contour part adopts the average weighting strategy for fusion， and the significant part adopts the VGG strategy for multi-layer fusion， and then the reconstructed image is fused； Finally， it is compared with the results of other algorithms， and the performance evaluation method is used to evaluate each fusion algorithm. The experimental results show that the algorithm can enhance the gray level of the scene information in the image， improve the brightness of the scene， and solve the problem of anti background interference of the scene information in the single frequency image.

Key words: infrared image, low-light image, low-pass filtering, VGG network, fusion algorithm

中图分类号: 

  • TP312

图1

红外与微光图像"

图2

三维图像对比"

图3

红外与微光图像直方图对比"

图4

红外与微光图像直方图均衡对比"

图5

红外与微光图像取反信息对比"

图 6

VGG19网络架构"

图7

红外与微光图像融合框架"

图8

基于VGG19的细节信息融合"

图9

图像1融合算法结果对比"

图10

图像2融合算法结果对比"

图11

6种算法的融合图像客观评价"

表1

性能评价对比"

评价算法信息熵（EN）标准差（SD）平均梯度（AG）结构相似度（SSIM）
VGG19与低通滤波（本文）7.749926.32014.57391.7267
主成分分析（PCA）6.233327.55673.96711.4723
小波变换（DWT）6.194524.15274.02131.6558
非下采样轮廓（NSCT）6.425.16283.72711.5129
生成对抗网络（GAN）7.162925.71263.81751.7064
残差网络（ResNet）6.952124.15834.35391.5665
1 Sharma A M, Dogra A, Goyal B, et al. Low-light visible and infrared image fusion in NSST domain[C]//Proceedings of International Conference on IoT Inclusive Life, Singapore, 2020: 61-68.
2 黄煜东. 真实场景的红外与微光图像融合方法研究[D].长沙: 国防科技大学智能科学学院,2018.
Huang Yi-dong. Fusion of infrared and low-level light images in realistic scene[D]. Changsha: College of Intelligent Sciences, National University of Defense Technology, 2003.
3 Zhang Z, Li H, Zhao G. Bionic algorithm for color fusion of infrared and low light level image based on rattlesnake bimodal cells[J]. IEEE Access, 2018, 6: 68981-68988.
4 Wang X, Yin J, Zhang K, et al. Infrared weak-small targets fusion based on latent low-rank representation and DWT[J]. IEEE Access, 2019, 7:112681-112692.
5 胡清平, 张晓晖, 刘超. 基于噪声评价的微光红外图像自适应融合方法[J]. 海军工程大学学报, 2017, 29(1): 102-106.
Hu Qing-ping, Zhang Xiao-hui, Liu Chao. Adaptive fusion method of low light level and infrared image based on noise analysis[J]. Journal of Naval University of Engineering, 2017, 29(1): 102-106.
6 Vijayarajan R, Nagarajan S, Karthik R, et al. Performance analysis of VGG19 deep learning network based Brain image fusion[M]//Handbook of Research on Deep Learning-Based Image Analysis Under Constrained and Unconstrained Environments, Hershey(USA): IGI Global, 2020: 145-166.
7 Yang L, Gomez-Garcia R, Munoz-Ferrer J M, et al. Input-reflectionless low-pass filter on multilayered diplexer-based topology[J]. IEEE Microwave and Wireless Components Letters, 2020, 30(10): 945-948.
8 Li H, Wu X J. Infrared and visible image fusion using a novel deep decomposition method [DB/OL].[2018-11-01]. .
9 Chen W B, Hu M, Zhou L, et al. Fusion algorithm of multi-focus images with weighted ratios and weighted gradient based on wavelet transform[J]. Journal of Intelligent Systems, 2019, 28(4): 505-516.
10 Nikbakhsh N, BaleghI Y, Agahi H. Maximum mutual information and Tsallis entropy for unsupervised segmentation of tree leaves in natural scenes[J]. Computers and Electronics in Agriculture, 2019, 162:440-449.
11 Yu B, Qiao T, Zhang H, et al. Dual band infrared detection method based on mid-infrared and long infrared vision forconveyor belts longitudinal tear[J]. Measurement, 2018, 120: 140-149.
12 Wang E, Yang B, Pang L. Super pixel-based structural similarity metric for image fusion quality evaluation[J]. Sensing and Imaging, 2021, 22(16): 1-25.
13 Nandal A, Rosales H G, et al. Modified PCA transformation with LWT for high-resolution based image fusion[J]. Iranian Journal of Science and Technology. 2019: S141-S157.
14 Sriwathi Nimmagadda, Nimmagadda Shastri L, Neel Mani. Design and development of a real time vision enhancement system using image fusion[J]. Procedia Computer Science, 2019, 159: 990-1000.
15 Liu Z, Y S, Sheng V S, et al. MRI and PET image fusion using the nonparametric density model and the theory of variable-weight[J]. Computer Methods & Programs in Biomedicine, 2019, 175: 73-82.
16 Yang Z, Chen Y, Le Z, et al. GAN fuse: a novel multi-exposure image fusion method based on generative adversarial networks[J]. Neural Computing and Applications, 2020,33: 6133-6145.
17 Li H, Wu X J, Durrani T S. Infrared and visible image fusion with resnet and zero-phase component analysis[J]. Infrared Physics & Technology, 2019, 102:No.103039.
[1] 刘玉梅,乔宁国,庄娇娇,刘鹏程,胡婷,陈立军. 基于多传感器数据融合的轨道车辆齿轮箱异常检测[J]. 吉林大学学报(工学版), 2019, 49(5): 1465-1470.
[2] 吴一全, 吴诗婳, 张宇飞. 基于混沌粒子群优化的Contourlet域红外图像自适应增强[J]. 吉林大学学报(工学版), 2014, 44(5): 1466-1473.
[3] 丁莹, 李文辉, 范静涛, 杨华民. 基于模糊积分特征的红外图像运动目标检测算法[J]. 吉林大学学报(工学版), 2010, 40(05): 1330-1335.
[4] 杨兆升, 杨庆芳, 冯金巧. 路段平均速度组合融合算法及其应用[J]. 吉林大学学报(工学版), 2004, (4): 675-678.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
版权所有 © 《吉林大学学报(工学版)》编辑部
地址：长春市人民大街5988号　电话：+86-431-85095297　传真：+86-431-85094074　E-mail: xbgxb@jlu.edu.cn
本系统由北京玛格泰克科技发展有限公司设计开发