基于相对总变差结构提取的遥感图像融合

doi:10.13229/j.cnki.jdxbgxb20200573

吉林大学学报(工学版) ›› 2021, Vol. 51 ›› Issue (5): 1775-1784.doi: 10.13229/j.cnki.jdxbgxb20200573

• 计算机科学与技术 • 上一篇

基于相对总变差结构提取的遥感图像融合

李雄飞^1,²(),吴佳婧^1,²,张小利^1,²(),王泽宇²,冯云丛^2,³

^1.吉林大学软件学院，长春 130012
^2.吉林大学符号计算与知识工程教育部重点实验室，长春 130012
^3.长春工业大学计算机科学与工程学院，长春 130012

收稿日期:2020-07-28 出版日期:2021-09-01 发布日期:2021-09-16
通讯作者: 张小利 E-mail:lxf@jlu.edu.cn;zhangxiaoli@jlu.edu.cn
作者简介:李雄飞（1963-），男，教授，博士生导师.研究方向：图像融合，机器学习.E-mail：lxf@jlu.edu.cn
基金资助:
国家自然科学基金项目(61801190);吉林省自然科学基金项目(20180101055JC);国家博士后科研基金项目(2017M611323);吉林省“十三五”教育科研规划项目(JJKH2020997KJ);中央高校基本科研基金项目(93K172020K05)

Remote sensing image fusion algorithm based on relative total variation structure extraction

Xiong-fei LI^1,²(),Jia-jing WU^1,²,Xiao-li ZHANG^1,²(),Ze-yu WANG²,Yun-cong FENG^2,³

^1.College of Software，Jilin University，Changchun 130012，China
^2.Key Laboratory of Symbolic Computation and Knowledge Engineering of Ministry of Education，Jilin University，Changchun 130012，China
^3.College of Computer Science and Engineering，Changchun University of Technology，Changchun 130012，China

Received:2020-07-28 Online:2021-09-01 Published:2021-09-16
Contact: Xiao-li ZHANG E-mail:lxf@jlu.edu.cn;zhangxiaoli@jlu.edu.cn

摘要/Abstract

摘要：

针对遥感图像融合结果光谱信息丰富度欠缺的问题，提出了一种基于结构提取的融合方法。首先，采用一种通过相对总变差从纹理中提取结构的方法对图像进行分解，分离出对应的低频和高频子带。其次，在高频层引入视觉决策图来指导高频进行融合，在低频融合规则中，同时考虑全色图像的边缘强度和多光谱的色彩信息强度来进行融合。最后，通过重构低频融合结果和高频融合结果得到融合图像。在Worldview-2卫星遥感图像上验证本文算法的有效性。实验表明：本文算法较好地保留了全色图像的细节信息以及多光谱图像的光谱信息，与其他同类算法相比具有一定的优越性。

关键词: 图像融合, 遥感图像, 结构提取, 融合规则

Abstract:

Aiming at taking both color information and edge retention into count as much as possible， an effective multi-scale fusion method based on structure extraction and visual salience map （VSM） is proposed. Firstly， the source images are decomposed by a validated structure extraction from the texture via relative total variation. Then， preserving the color of MS and the edge definition are considered in the fusion of base layers and the conventional 'averaging' fusion scheme is unable to achieve the same effects. Next， VSM-based technique is proposed to fuse the detail layers to figure out perceptually and combine salient visual structures， standing out from neighbors. Finally， the desired fusion image was obtained by image reconstruction， using the detailed and based fused results. Remote sensing images from the Worldview-2 satellite are used as the data set which verifies the effectiveness of our algorithm. Experiments show that the algorithm designed in this paper retains the detail information of the panchromatic image and the spectral information of multispectral image， and highlight the expediency and suitability with comparing with others.

Key words: image fusion, remote sensing images, structure extraction, fusion rules

中图分类号:

TP391

李雄飞,吴佳婧,张小利,王泽宇,冯云丛. 基于相对总变差结构提取的遥感图像融合[J]. 吉林大学学报(工学版), 2021, 51(5): 1775-1784.

Xiong-fei LI,Jia-jing WU,Xiao-li ZHANG,Ze-yu WANG,Yun-cong FENG. Remote sensing image fusion algorithm based on relative total variation structure extraction[J]. Journal of Jilin University(Engineering and Technology Edition), 2021, 51(5): 1775-1784.

图/表 11

图1

图2

图3

图4

图5

表1

表2

表3

图6

表4

表5

参考文献 23

1	Liu X Y, Liu Q J, Wang Y H. Remote sensing image fusion based on two-stream fusion network[J]. Information Fusion, 2020, 55: 1-15.
2	Sun J. A study of PCA image fusion techniques on remote sensing[C]∥International Conference on Space Information Technology. International Society for Optics and Photonics, 2005: 739-744.
3	Hassan G. A review of remote sensing image fusion methods[J]. Information Fusion, 2016, 32: 75-89.
4	Leung Y, Liu J, Zhang J. An improved adaptive intensity-hue-saturation method for the fusion of remote sensing images[J]. IEEE Geoscience & Remote Sensing Letters, 2013, 11(5): 985-989.
5	包磊, 徐其志. 基于PCA变换和光谱补偿的遥感影像融合方法[J]. 吉林大学学报: 工学版, 2013, 43(): 88-91.
	Bao Lei, Xu Qi-zhi. Spectrum-keeping algorithm for fusing based on PCA[J]. Journal of Jilin University(Engineering and Technology Edition), 2013, 43(Sup.1): 88-91.
6	Tu T M, Lee Y C, Chang C P, et al. Adjustable intensity-hue-saturation and Brovey transform fusion technique for IKONOS/QuickBird imagery[J].Optical Engineering, 2005, 44(11): 116201.
7	Candra D S. Analysis of SPOT-6 data fusion using Gram-Schmidt spectral sharpening on rural areas[J]. International Journal of Remote Sensing & Earth Sciences, 2013, 10(2): 84-89.
8	Shahdoosti H R, Ghassemian H. Spatial PCA as a new method for image fusion[C]∥Csi International Symposium on Artificial Intelligence and Signal Processing,Shiraz, Iran, 2012: 90-94.
9	陈荣元, 郑晨, 申立智. 多源影像融合与分割的协同方法[J]. 电子学报, 2015, 43(10): 1994-2000.
	Chen Rong-yuan, Zheng Chen, Shen Li-zhi.Cooperation between fusion and segmentation for multisource image[J]. Acta Electronica Sinica, 2015, 43(10): 1994-2000.
10	李雄飞, 宋璐, 张小利. 基于协同经验小波变换的遥感图像融合[J]. 吉林大学学报: 工学版, 2019, 49(4): 1307-1319.
	Li Xiong-fei, Song Lu, Zhang Xiao-li. Remote sensing image fusion based on cooperative empirical wavelet transform[J]. Journal of Jilin University(Engineering and Technology Edition), 2019, 49(4): 1307-1319.
11	金海燕, 焦李成. 基于CP和多小波HMT模型的克隆选择遥感图像融合[J]. 计算机学报, 2009, 32(7): 1434-1442.
	Jin Hai-yan, Jiao Li-cheng. Clonal selection remote sensing image fusion based on CP and multiwavelet HMT models[J]. Chinese Journal of Computers, 2009, 32(7): 1434-1442.
12	Li Y T, Wu J, Yang Y. HIS and DWT based remote sensing image fusion and its application on road extraction[C]∥2011 International Conference on Consumer Electronics, Communications and Networks, Xianning, 2011, 394-397.
13	李光鑫, 王珂. 基于Contourlet变换的彩色图像融合算法[J]. 电子学报, 2007, 35(1): 112-117.
	Li Guang-xin, Wang Ke. Color image fusion algorithm using the contourlet transform[J]. Acta Electronica Sinica, 2007,35(1): 112-117.
14	Li T J, Wang Y Y. Biological image fusion using a NSCT based variable weight method[J]. Information Fusion, 2011, 12(2): 85-92.
15	Li H, Zhang F, Zhang K. Remote sensing image fusion based on sparse non-negative matrix factorization[J]. Journal of Xidian University, 2016, 43(2): 193-198.
16	Dong Y N, Du B, Zhang L P. Target detection based on random forest metric learning[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2017, 8(4): 1830-1838.
17	He G, Ji J, Dong D, et al. Infrared and visible image fusion method by using hybrid representation learning[J]. Geoscience and Remote Sensing Letters, 2019, 6(11): 1796-1800.
18	Xiao A, Wang Z Y, Wang L, et al. Super-resolution for 'Jilin-1' satellite video imagery via a convolutional network[J]. Sensors, 2018, 18(4): 18041194.
19	Liu Y, Chen X, Wang Z, et al. Deep learning for pixel-level image fusion: recent advances and future prospects[J]. Information Fusion, 2018, 42: 158-173.
20	Xu L, Yan Q, Xia Y, et al. Structure extraction from texture via relative total variation[J]. ACM Transaction on Graphics, 2012, 31(6): 2366158.
21	Ma J L, Zhou Z Q, Wang B, et al. Infrared and visible image fusion based on visual saliency map and weighted least square optimization[J]. Infrared Physics and Technology, 2017, 82: 8-17.
22	Tan W, Xiang P, Zhang J, et al. Remote sensing image fusion via boundary measured dual-channel PCNN in multi-scale morphological gradient domain[J]. Access, 2020, 99: 42540-42549.
23	Liu Y, Chen X, Cheng J,et al. A medical image fusion method based on convolutional neural networks[C]∥20th International Conference on Information Fusion, Xi'an, China, 2017: 17102534.

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed

对比算法	RMSE	ERGAS	PSNR	CC	SAM	SSIM
CVT	13.4862	29.4823	11.2168	0.9503	0.1517	0.9124
IHS	17.1032	29.1758	9.5822	0.9298	0.1518	0.9292
DWT	13.7324	30.3887	11.3545	0.9506	0.1592	0.8937
NSCT	13.1217	28.9521	11.4420	0.9537	0.1491	0.8970
CMP	15.3394	28.9761	9.9934	0.9319	0.1799	0.8663
CNN	13.2136	27.9425	9.7078	0.9529	0.1595	0.8941
本文算法	12.9425	28.9390	11.5551	0.9547	0.1477	0.9415

对比算法	RMSE	ERGAS	PSNR	CC	SAM	SSIM
CVT	12.0094	34.3946	11.8553	0.9547	0.3625	0.7233
IHS	14.5034	31.2119	10.2903	0.9474	0.3603	0.7803
DWT	12.4123	36.0190	11.8298	0.9536	0.3647	0.7244
NSCT	11.8419	34.0795	11.9622	0.9576	0.3633	0.7393
CMP	17.8929	56.4643	11.5213	0.9113	0.3750	0.8096
CNN	14.2094	44.1196	10.3727	0.9532	0.3635	0.7593
本文算法	11.5473	31.1068	13.1554	0.9578	0.3587	0.7989

对比算法	RMSE	ERGAS	PSNR	CC	SAM	SSIM
CVT	15.4473	50.4017	10.7316	0.9119	0.2309	0.8831
IHS	23.4215	61.7238	8.2385	0.9022	0.2174	0.8595
DWT	15.7092	50.8902	10.7146	0.9132	0.2374	0.8761
NSCT	15.3834	50.2933	10.7444	0.9157	0.2284	0.8865
CMP	20.9036	57.0532	10.1177	0.8929	0.2581	0.8381
CNN	22.7081	60.7507	8.3688	0.9043	0.2171	0.8891
本文算法	15.3902	43.5187	13.3128	0.9168	0.2164	0.9016

对比算法	RMSE	ERGAS	PSNR	CC	SAM	SSIM
平均值法	56.8854	55.8427	5.0612	0.8866	0.3013	0.9004
加权最小二乘法	13.1782	34.7411	10.4485	0.9453	0.2488	0.8848
局部能量法	16.3256	39.4121	8.5692	0.9233	0.2817	0.8913
本文算法	12.6608	32.8150	12.7963	0.9461	0.2164	0.9016

对比算法	RMSE	ERGAS	PSNR	CC	SAM	SSIM
绝对值取最大策略	13.0726	33.1260	11.8636	0.9102	0.2542	0.8926
局部能量法	14.1522	37.8828	10.2385	0.9002	0.2263	0.8733
加权最小二乘法	14.0767	34.8610	12.4006	0.9359	0.2348	0.8845
本文算法	12.6608	32.8150	12.7963	0.9461	0.2164	0.9016

基于相对总变差结构提取的遥感图像融合

Remote sensing image fusion algorithm based on relative total variation structure extraction

RICH HTML

PDF (PC)

摘要/Abstract

引用本文

使用本文

图/表 11

参考文献 23

相关文章 15

Metrics

本文评价

推荐阅读 0

[1]	陈广秋,陈昱存,李佳悦,刘广文. 基于DNST和卷积稀疏表示的红外与可见光图像融合[J]. 吉林大学学报(工学版), 2021, 51(3): 996-1010.
[2]	陈蔓,钟勇,李振东. 隐低秩结合低秩表示的多聚焦图像融合[J]. 吉林大学学报(工学版), 2020, 50(1): 297-305.
[3]	李雄飞,王婧,张小利,范铁虎. 基于SVM和窗口梯度的多焦距图像融合方法[J]. 吉林大学学报(工学版), 2020, 50(1): 227-236.
[4]	李雄飞,宋璐,张小利. 基于协同经验小波变换的遥感图像融合[J]. 吉林大学学报(工学版), 2019, 49(4): 1307-1319.
[5]	刘哲, 徐涛, 宋余庆, 徐春艳. 基于NSCT变换和相似信息鲁棒主成分分析模型的图像融合技术[J]. 吉林大学学报(工学版), 2018, 48(5): 1614-1620.
[6]	尹明, 战荫伟, 裴海龙. 基于稀疏补算子学习的图像融合方法[J]. 吉林大学学报(工学版), 2016, 46(6): 2052-2058.
[7]	于德新, 刘春宇, 郑坤, 郭亚娟, 赵鑫. 基于遥感图像的车辆空间占有率估计[J]. 吉林大学学报(工学版), 2016, 46(3): 764-769.
[8]	王晓文^{1, 2}, 赵宗贵³, 庞秀梅⁴, 刘敏⁵. 图的图像融合方法[J]. 吉林大学学报(工学版), 2014, 44(4): 1203-1208.
[9]	杨扬, 戴明, 周箩鱼, 孙明超. 基于非下采样Bandelet变换的多聚焦图像融合[J]. 吉林大学学报(工学版), 2014, 44(2): 525-530.
[10]	王昕, 李玮琳, 刘富. 小波域CT/MRI医学图像融合新方法[J]. 吉林大学学报(工学版), 2013, 43(增刊1): 25-28.
[11]	李娟, 南旭良, 毕思远, 吴微. 改进多目标粒子群优化算法及在图像融合中的应用[J]. 吉林大学学报(工学版), 2013, 43(增刊1): 477-480.
[12]	包磊, 徐其志. 基于PCA变换和光谱补偿的遥感影像融合方法[J]. 吉林大学学报(工学版), 2013, 43(增刊1): 88-91.
[13]	冯鑫, 王晓明, 党建武, 沈瑜. 基于插值Directionlet变换的图像融合方法[J]. 吉林大学学报(工学版), 2013, 43(04): 1127-1132.
[14]	邢笑雪, 刘富, 马冬梅, 翟微微, 王芳荣. 基于机器视觉的软体纤维丝集束智能计数系统[J]. , 2012, 42(05): 1267-1272.
[15]	刘刚, 赵红毅, 胡臻龙. 基于带有权值混合泛函的盲超分辨率[J]. , 2012, 42(04): 1054-1058.