基于相对总变差结构提取的遥感图像融合

doi:10.13229/j.cnki.jdxbgxb20200573

Abstract

Abstract:

Aiming at taking both color information and edge retention into count as much as possible， an effective multi-scale fusion method based on structure extraction and visual salience map （VSM） is proposed. Firstly， the source images are decomposed by a validated structure extraction from the texture via relative total variation. Then， preserving the color of MS and the edge definition are considered in the fusion of base layers and the conventional 'averaging' fusion scheme is unable to achieve the same effects. Next， VSM-based technique is proposed to fuse the detail layers to figure out perceptually and combine salient visual structures， standing out from neighbors. Finally， the desired fusion image was obtained by image reconstruction， using the detailed and based fused results. Remote sensing images from the Worldview-2 satellite are used as the data set which verifies the effectiveness of our algorithm. Experiments show that the algorithm designed in this paper retains the detail information of the panchromatic image and the spectral information of multispectral image， and highlight the expediency and suitability with comparing with others.

Key words: image fusion, remote sensing images, structure extraction, fusion rules

CLC Number:

TP391

Xiong-fei LI,Jia-jing WU,Xiao-li ZHANG,Ze-yu WANG,Yun-cong FENG. Remote sensing image fusion algorithm based on relative total variation structure extraction[J].Journal of Jilin University(Engineering and Technology Edition), 2021, 51(5): 1775-1784.

Figures/Tables 11

Fig.1

Fig.2

Fig.3

Fig.4

Fig.5

Table 1

Table 2

Table 3

Fig.6

Table 4

Table 5

References 23

1	Liu X Y, Liu Q J, Wang Y H. Remote sensing image fusion based on two-stream fusion network[J]. Information Fusion, 2020, 55: 1-15.
2	Sun J. A study of PCA image fusion techniques on remote sensing[C]∥International Conference on Space Information Technology. International Society for Optics and Photonics, 2005: 739-744.
3	Hassan G. A review of remote sensing image fusion methods[J]. Information Fusion, 2016, 32: 75-89.
4	Leung Y, Liu J, Zhang J. An improved adaptive intensity-hue-saturation method for the fusion of remote sensing images[J]. IEEE Geoscience & Remote Sensing Letters, 2013, 11(5): 985-989.
5	包磊, 徐其志. 基于PCA变换和光谱补偿的遥感影像融合方法[J]. 吉林大学学报: 工学版, 2013, 43(): 88-91.
	Bao Lei, Xu Qi-zhi. Spectrum-keeping algorithm for fusing based on PCA[J]. Journal of Jilin University(Engineering and Technology Edition), 2013, 43(Sup.1): 88-91.
6	Tu T M, Lee Y C, Chang C P, et al. Adjustable intensity-hue-saturation and Brovey transform fusion technique for IKONOS/QuickBird imagery[J].Optical Engineering, 2005, 44(11): 116201.
7	Candra D S. Analysis of SPOT-6 data fusion using Gram-Schmidt spectral sharpening on rural areas[J]. International Journal of Remote Sensing & Earth Sciences, 2013, 10(2): 84-89.
8	Shahdoosti H R, Ghassemian H. Spatial PCA as a new method for image fusion[C]∥Csi International Symposium on Artificial Intelligence and Signal Processing,Shiraz, Iran, 2012: 90-94.
9	陈荣元, 郑晨, 申立智. 多源影像融合与分割的协同方法[J]. 电子学报, 2015, 43(10): 1994-2000.
	Chen Rong-yuan, Zheng Chen, Shen Li-zhi.Cooperation between fusion and segmentation for multisource image[J]. Acta Electronica Sinica, 2015, 43(10): 1994-2000.
10	李雄飞, 宋璐, 张小利. 基于协同经验小波变换的遥感图像融合[J]. 吉林大学学报: 工学版, 2019, 49(4): 1307-1319.
	Li Xiong-fei, Song Lu, Zhang Xiao-li. Remote sensing image fusion based on cooperative empirical wavelet transform[J]. Journal of Jilin University(Engineering and Technology Edition), 2019, 49(4): 1307-1319.
11	金海燕, 焦李成. 基于CP和多小波HMT模型的克隆选择遥感图像融合[J]. 计算机学报, 2009, 32(7): 1434-1442.
	Jin Hai-yan, Jiao Li-cheng. Clonal selection remote sensing image fusion based on CP and multiwavelet HMT models[J]. Chinese Journal of Computers, 2009, 32(7): 1434-1442.
12	Li Y T, Wu J, Yang Y. HIS and DWT based remote sensing image fusion and its application on road extraction[C]∥2011 International Conference on Consumer Electronics, Communications and Networks, Xianning, 2011, 394-397.
13	李光鑫, 王珂. 基于Contourlet变换的彩色图像融合算法[J]. 电子学报, 2007, 35(1): 112-117.
	Li Guang-xin, Wang Ke. Color image fusion algorithm using the contourlet transform[J]. Acta Electronica Sinica, 2007,35(1): 112-117.
14	Li T J, Wang Y Y. Biological image fusion using a NSCT based variable weight method[J]. Information Fusion, 2011, 12(2): 85-92.
15	Li H, Zhang F, Zhang K. Remote sensing image fusion based on sparse non-negative matrix factorization[J]. Journal of Xidian University, 2016, 43(2): 193-198.
16	Dong Y N, Du B, Zhang L P. Target detection based on random forest metric learning[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2017, 8(4): 1830-1838.
17	He G, Ji J, Dong D, et al. Infrared and visible image fusion method by using hybrid representation learning[J]. Geoscience and Remote Sensing Letters, 2019, 6(11): 1796-1800.
18	Xiao A, Wang Z Y, Wang L, et al. Super-resolution for 'Jilin-1' satellite video imagery via a convolutional network[J]. Sensors, 2018, 18(4): 18041194.
19	Liu Y, Chen X, Wang Z, et al. Deep learning for pixel-level image fusion: recent advances and future prospects[J]. Information Fusion, 2018, 42: 158-173.
20	Xu L, Yan Q, Xia Y, et al. Structure extraction from texture via relative total variation[J]. ACM Transaction on Graphics, 2012, 31(6): 2366158.
21	Ma J L, Zhou Z Q, Wang B, et al. Infrared and visible image fusion based on visual saliency map and weighted least square optimization[J]. Infrared Physics and Technology, 2017, 82: 8-17.
22	Tan W, Xiang P, Zhang J, et al. Remote sensing image fusion via boundary measured dual-channel PCNN in multi-scale morphological gradient domain[J]. Access, 2020, 99: 42540-42549.
23	Liu Y, Chen X, Cheng J,et al. A medical image fusion method based on convolutional neural networks[C]∥20th International Conference on Information Fusion, Xi'an, China, 2017: 17102534.

Related Articles 15

[1]	Guang-qiu CHEN,Yu-cun CHEN,Jia-yue LI,Guang-wen LIU. Infrared and visible image fusion based on discrete nonseparable shearlet transform and convolutional sparse representation [J]. Journal of Jilin University(Engineering and Technology Edition), 2021, 51(3): 996-1010.
[2]	Man CHEN,Yong ZHONG,Zhen-dong LI. Multi-focus image fusion based on latent low⁃rank representation combining low⁃rank representation [J]. Journal of Jilin University(Engineering and Technology Edition), 2020, 50(1): 297-305.
[3]	Xiong-fei LI,Jing WANG,Xiao-li ZHANG,Tie-hu FAN. Multi-focus image fusion based on support vector machines and window gradient [J]. Journal of Jilin University(Engineering and Technology Edition), 2020, 50(1): 227-236.
[4]	Xiong-fei LI,Lu SONG,Xiao-li ZHANG. Remote sensing image fusion based on cooperative empirical wavelet transform [J]. Journal of Jilin University(Engineering and Technology Edition), 2019, 49(4): 1307-1319.
[5]	LIU Zhe, XU Tao, SONG Yu-qing, XU Chun-yan. Image fusion technology based on NSCT and robust principal component analysis model with similar information [J]. Journal of Jilin University(Engineering and Technology Edition), 2018, 48(5): 1614-1620.
[6]	YIN Ming, ZHAN Yin-wei, PEI Hai-long. Co-sparse analysis operator learning for image fusion [J]. 吉林大学学报(工学版), 2016, 46(6): 2052-2058.
[7]	YU De-xin, LIU Chun-yu, ZHENG Kun, GUO Ya-juan, ZHAO Xin. Space occupancy estimating method based on remote sensing images [J]. 吉林大学学报(工学版), 2016, 46(3): 764-769.
[8]	WANG Xiao-wen, ZHAO Zong-gui, PANG Xiu-mei, LIU Min. Image fusion method based on visual saliency maps [J]. 吉林大学学报(工学版), 2014, 44(4): 1203-1208.
[9]	YANG Yang, DAI Ming, ZHOU Luo-yu, SUN Ming-chao. Multifocus image fusion based on nonsubsampled Bandelet transform [J]. 吉林大学学报(工学版), 2014, 44(2): 525-530.
[10]	WANG Xin, LI Wei-lin, LIU Fu. New algorithm of CT/MRI medical image fusion based on wavelet domain [J]. 吉林大学学报(工学版), 2013, 43(增刊1): 25-28.
[11]	LI Juan, NAN Xu-liang, BI Si-yuan, WU Wei. Image Fusion based on an improved algorithm of Multi-objective Particle swarm Optimization [J]. 吉林大学学报(工学版), 2013, 43(增刊1): 477-480.
[12]	BAO Lei, XU Qi-zhi. Spectrum-keeping algorithm for fusing based on PCA [J]. 吉林大学学报(工学版), 2013, 43(增刊1): 88-91.
[13]	XING Xiao-xue, LIU Fu, MA Dong-mei, ZHAI Wei-wei, WANG Fang-rong. Intelligent counting system of soft fibrils collection based on machine vision [J]. , 2012, 42(05): 1267-1272.
[14]	LIU Gang, ZHAO Hong-yi, HU Zhen-long. Weighted mixed-norm based blind super-resolution algorithm [J]. , 2012, 42(04): 1054-1058.
[15]	SUN Ming-chao, ZHANG Chong, LIU Jing-hong. Fusion of visible and infrared images based on multi-scale image enhancement [J]. , 2012, (03): 738-742.

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed

Comments

Recommended 0

No Suggested Reading articles found!

对比算法	RMSE	ERGAS	PSNR	CC	SAM	SSIM
CVT	13.4862	29.4823	11.2168	0.9503	0.1517	0.9124
IHS	17.1032	29.1758	9.5822	0.9298	0.1518	0.9292
DWT	13.7324	30.3887	11.3545	0.9506	0.1592	0.8937
NSCT	13.1217	28.9521	11.4420	0.9537	0.1491	0.8970
CMP	15.3394	28.9761	9.9934	0.9319	0.1799	0.8663
CNN	13.2136	27.9425	9.7078	0.9529	0.1595	0.8941
本文算法	12.9425	28.9390	11.5551	0.9547	0.1477	0.9415

对比算法	RMSE	ERGAS	PSNR	CC	SAM	SSIM
CVT	12.0094	34.3946	11.8553	0.9547	0.3625	0.7233
IHS	14.5034	31.2119	10.2903	0.9474	0.3603	0.7803
DWT	12.4123	36.0190	11.8298	0.9536	0.3647	0.7244
NSCT	11.8419	34.0795	11.9622	0.9576	0.3633	0.7393
CMP	17.8929	56.4643	11.5213	0.9113	0.3750	0.8096
CNN	14.2094	44.1196	10.3727	0.9532	0.3635	0.7593
本文算法	11.5473	31.1068	13.1554	0.9578	0.3587	0.7989

对比算法	RMSE	ERGAS	PSNR	CC	SAM	SSIM
CVT	15.4473	50.4017	10.7316	0.9119	0.2309	0.8831
IHS	23.4215	61.7238	8.2385	0.9022	0.2174	0.8595
DWT	15.7092	50.8902	10.7146	0.9132	0.2374	0.8761
NSCT	15.3834	50.2933	10.7444	0.9157	0.2284	0.8865
CMP	20.9036	57.0532	10.1177	0.8929	0.2581	0.8381
CNN	22.7081	60.7507	8.3688	0.9043	0.2171	0.8891
本文算法	15.3902	43.5187	13.3128	0.9168	0.2164	0.9016

对比算法	RMSE	ERGAS	PSNR	CC	SAM	SSIM
平均值法	56.8854	55.8427	5.0612	0.8866	0.3013	0.9004
加权最小二乘法	13.1782	34.7411	10.4485	0.9453	0.2488	0.8848
局部能量法	16.3256	39.4121	8.5692	0.9233	0.2817	0.8913
本文算法	12.6608	32.8150	12.7963	0.9461	0.2164	0.9016

对比算法	RMSE	ERGAS	PSNR	CC	SAM	SSIM
绝对值取最大策略	13.0726	33.1260	11.8636	0.9102	0.2542	0.8926
局部能量法	14.1522	37.8828	10.2385	0.9002	0.2263	0.8733
加权最小二乘法	14.0767	34.8610	12.4006	0.9359	0.2348	0.8845
本文算法	12.6608	32.8150	12.7963	0.9461	0.2164	0.9016

Remote sensing image fusion algorithm based on relative total variation structure extraction

RICH HTML

PDF (PC)