Journal of Jilin University(Engineering and Technology Edition) ›› 2023, Vol. 53 ›› Issue (1): 255-262.doi: 10.13229/j.cnki.jdxbgxb20210542

Previous Articles    

Infrared and low⁃light image fusion based on VGG19 and low⁃pass filtering

Zhou-zhou LIU1(),Chuan-xin SUN2,Xiao-zhu WANG3,Yang-mei ZHANG4   

  1. 1.School of Computer Science，Xi'an Aeronautical University，Xi'an 710077，China
    2.No. 29 Research Institute of CETC，Chengdu 610036，China
    3.School of Data Sciences，The Chinese University of Hong Kong（Shenzhen），Shenzhen 518172，China
    4.School of Electronic Engineering，Xi'an Aeronautical University，Xi'an 710077，China
  • Received:2021-06-18 Online:2023-01-01 Published:2023-07-23

RICH HTML

 2   

PDF (PC)

135

Abstract:

Due to the current single sensor being affected by its imaging performance， it is often difficult to fully reflect all the effective information in the ground object scene， resulting in the problem that the scene information is difficult to accurately identify， so a new infrared and low light image fusion algorithm based on VGG and low-pass filtering is proposed. Firstly， obtain the ground object scene information through infrared and low-level light detectors， use three-dimensional distribution， histogram comparison， and inversion to process the image， analyze the target characteristics of infrared and low-level light images， and study the spectroscopic mechanism of dual-frequency images； On this basis， the low-pass filtering method is used to decompose the infrared and low-light images to obtain their contour information and salient information. The contour part adopts the average weighting strategy for fusion， and the significant part adopts the VGG strategy for multi-layer fusion， and then the reconstructed image is fused； Finally， it is compared with the results of other algorithms， and the performance evaluation method is used to evaluate each fusion algorithm. The experimental results show that the algorithm can enhance the gray level of the scene information in the image， improve the brightness of the scene， and solve the problem of anti background interference of the scene information in the single frequency image.

Key words: infrared image, low-light image, low-pass filtering, VGG network, fusion algorithm

CLC Number: 

  • TP312

Fig.1

Infrared image and low-light image"

Fig.2

Three-dimensional distributions of the Infrared image and low-light image"

Fig.3

Histgramof the Infrared image and low-light image"

Fig.4

Histgram equalizationof the Infrared image and low-light image"

Fig.5

Image inversion of the infrared image and low-light image"

Fig.6

VGG19 network architecture"

Fig.7

Workflow for infrared and low-light image fusion"

Fig. 8

Fusion of the detailed information based on VGG19"

Fig.9

Comparison of image 1 fusion algorithm results"

Fig.10

Comparison of image 1 fusion algorithm results"

Fig.11

Objective evaluation of fused images based on six algorithms"

Table 1

Comparison of algorithm performances"

评价算法信息熵（EN）标准差（SD）平均梯度（AG）结构相似度（SSIM）
VGG19与低通滤波（本文）7.749926.32014.57391.7267
主成分分析（PCA）6.233327.55673.96711.4723
小波变换（DWT）6.194524.15274.02131.6558
非下采样轮廓（NSCT）6.425.16283.72711.5129
生成对抗网络（GAN）7.162925.71263.81751.7064
残差网络（ResNet）6.952124.15834.35391.5665
1 Sharma A M, Dogra A, Goyal B, et al. Low-light visible and infrared image fusion in NSST domain[C]//Proceedings of International Conference on IoT Inclusive Life, Singapore, 2020: 61-68.
2 黄煜东. 真实场景的红外与微光图像融合方法研究[D].长沙: 国防科技大学智能科学学院,2018.
Huang Yi-dong. Fusion of infrared and low-level light images in realistic scene[D]. Changsha: College of Intelligent Sciences, National University of Defense Technology, 2003.
3 Zhang Z, Li H, Zhao G. Bionic algorithm for color fusion of infrared and low light level image based on rattlesnake bimodal cells[J]. IEEE Access, 2018, 6: 68981-68988.
4 Wang X, Yin J, Zhang K, et al. Infrared weak-small targets fusion based on latent low-rank representation and DWT[J]. IEEE Access, 2019, 7:112681-112692.
5 胡清平, 张晓晖, 刘超. 基于噪声评价的微光红外图像自适应融合方法[J]. 海军工程大学学报, 2017, 29(1): 102-106.
Hu Qing-ping, Zhang Xiao-hui, Liu Chao. Adaptive fusion method of low light level and infrared image based on noise analysis[J]. Journal of Naval University of Engineering, 2017, 29(1): 102-106.
6 Vijayarajan R, Nagarajan S, Karthik R, et al. Performance analysis of VGG19 deep learning network based Brain image fusion[M]//Handbook of Research on Deep Learning-Based Image Analysis Under Constrained and Unconstrained Environments, Hershey(USA): IGI Global, 2020: 145-166.
7 Yang L, Gomez-Garcia R, Munoz-Ferrer J M, et al. Input-reflectionless low-pass filter on multilayered diplexer-based topology[J]. IEEE Microwave and Wireless Components Letters, 2020, 30(10): 945-948.
8 Li H, Wu X J. Infrared and visible image fusion using a novel deep decomposition method [DB/OL].[2018-11-01]. .
9 Chen W B, Hu M, Zhou L, et al. Fusion algorithm of multi-focus images with weighted ratios and weighted gradient based on wavelet transform[J]. Journal of Intelligent Systems, 2019, 28(4): 505-516.
10 Nikbakhsh N, BaleghI Y, Agahi H. Maximum mutual information and Tsallis entropy for unsupervised segmentation of tree leaves in natural scenes[J]. Computers and Electronics in Agriculture, 2019, 162:440-449.
11 Yu B, Qiao T, Zhang H, et al. Dual band infrared detection method based on mid-infrared and long infrared vision forconveyor belts longitudinal tear[J]. Measurement, 2018, 120: 140-149.
12 Wang E, Yang B, Pang L. Super pixel-based structural similarity metric for image fusion quality evaluation[J]. Sensing and Imaging, 2021, 22(16): 1-25.
13 Nandal A, Rosales H G, et al. Modified PCA transformation with LWT for high-resolution based image fusion[J]. Iranian Journal of Science and Technology. 2019: S141-S157.
14 Sriwathi Nimmagadda, Nimmagadda Shastri L, Neel Mani. Design and development of a real time vision enhancement system using image fusion[J]. Procedia Computer Science, 2019, 159: 990-1000.
15 Liu Z, Y S, Sheng V S, et al. MRI and PET image fusion using the nonparametric density model and the theory of variable-weight[J]. Computer Methods & Programs in Biomedicine, 2019, 175: 73-82.
16 Yang Z, Chen Y, Le Z, et al. GAN fuse: a novel multi-exposure image fusion method based on generative adversarial networks[J]. Neural Computing and Applications, 2020,33: 6133-6145.
17 Li H, Wu X J, Durrani T S. Infrared and visible image fusion with resnet and zero-phase component analysis[J]. Infrared Physics & Technology, 2019, 102:No.103039.
[1] Yu-mei LIU,Ning-guo QIAO,Jiao-jiao ZHUANG,Peng-cheng LIU,Ting HU,Li-jun CHEN. Anomaly detection of rail vehicle gearbox based on multi⁃sensor data fusion [J]. Journal of Jilin University(Engineering and Technology Edition), 2019, 49(5): 1465-1470.
[2] WU Yi-quan,WU Shi-hua,ZHANG Yu-fei. Infrared image adaptive enhancement in Contourlet domain based on chaotic particle swarm optimization [J]. 吉林大学学报(工学版), 2014, 44(5): 1466-1473.
[3] DING Ying, LI Wen-Hui, FAN Jing-Tao, YANG Hua-Min. Fuzzy integral feature based algorithm for moving infrared object detection [J]. 吉林大学学报(工学版), 2010, 40(05): 1330-1335.
[4] YANG Zhaosheng, YANG Qingfang, FENG Jinqiao. Combined time-space average speed information fusion algorithm and its application [J]. 吉林大学学报(工学版), 2004, (4): 675-678.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
Copyright © Journal of Jilin University(Engineering and Technology Edition), All Rights Reserved.
Tel: +86-431-85095297　Fax: +86-431-85094074　E-mail: xbgxb@jlu.edu.cn
Powered by Beijing Magtech Co. Ltd