基于多层次空⁃谱融合网络的高光谱图像分类

doi:10.13229/j.cnki.jdxbgxb20210298

摘要/Abstract

摘要：

为了在高光谱图像分类中更好提取和表达光谱与空间的精细特征以及特征间的交互信息，提出一种基于多层次空-谱融合网络的高光谱图像分类方法。首先，利用多层次特征提取模块，分别提取高光谱图像的多层次空间和光谱特征；其次，设计空-谱特征交互融合模块将获得的多层次空间与光谱特征进行特征融合，以产生空-谱融合特征。本文方法可以结合网络中不同层次的空间与光谱特征，有效地捕获高光谱图像精细特征；同时，通过联合学习融合空间与光谱特征，捕获光谱与空间特征之间交互作用。实验结果表明，与现有基于神经网络的分类方法相比，所提出的高光谱图像分类算法能够获得更高的分类精度，表明该网络能有效地提取精细特征和增强空-谱融合特征的表达能力。

关键词: 高光谱图像分类, 多层次特征提取模块, 空?谱特征交互融合模块, 特征融合

Abstract:

To acquire fine spectral and spatial features and their interaction information in hyperspectral image classification， a hierarchical spatial-spectral fusion network is proposed. Firstly， the hierarchical features extraction module was exploited to extract the spectral and spatial features of hyperspectral image respectively. Secondly， the spatial-spectral feature interactive fusion module was designed and employed to fuse the features and produce the joint spatial-spectral features. The proposed network can not only extract and integrate the fine spatial and spectral features of different levels，but also capture the interaction between spectral and spatial features by joint learning. The experimental results show that the proposed network performed better than the state-of-the-art neural network-based classification methods. The network is shown the capability of extracting fine features and capturing the spatial-spectral joint features for classification.

Key words: hyperspectral image classification, hierarchical features extraction module, spatial-spectral feature interactive fusion module, feature fusion

中图分类号:

TP753

欧阳宁,李祖锋,林乐平. 基于多层次空⁃谱融合网络的高光谱图像分类[J]. 吉林大学学报(工学版), 2022, 52(10): 2438-2446.

Ning OUYANG,Zu-feng LI,Le-ping LIN. Hyperspectral image classification based on hierarchical spatial-spectral fusion network[J]. Journal of Jilin University(Engineering and Technology Edition), 2022, 52(10): 2438-2446.

图/表 12

图1

图2

表1

空间-光谱多层次特征提取模块参数设置"

操作类型	HFEM_spa	HFEM_spe
步幅卷积size /strides	$2 × 2 × 1,12 2 × 2 × 1, s t r i d e s$	$1 × 1 × 2,12 1 × 1 × 2, s t r i d e s$
常规卷积size /strides	$2 × 2 × 1,12 1 × 1 × 1, s t r i d e s$	$1 × 1 × 2,12 1 × 1 × 1, s t r i d e s$
Conv size /strides	$2 × 2 × 2,512 1 × 1 × 1, s t r i d e s$	$1 × 1 × 2,512 1 × 1 × 1, s t r i d e s$
Reshape	$512 × S 1$	$512 × S 2$

表1

图3

表2

表3

表4

表5

图4

图5

图6

表6

参考文献 20

1	Li W, Prasad S, Flower J E, et al. Locality-preserving dimensionality reduction and classification for hyperspectral image analysis[J]. IEEE Transactions on Geoscience & Remote Sensing,2012,50(4):1185-1198.
2	Li W, Chen C, Su H J, et al. Local binary patterns and extreme learning machine for hyperspectral imagery classification[J]. IEEE Transactions on Geoence & Remote Sensing, 2015, 53(7): 3681-3693.
3	Huang K, Li S, Kang X, et al. Spectral-spatial hyperspectral image classification based on KNN[J]. Sensing and Imaging, 2016, 17(1):1-13.
4	闫敬文, 陈宏达, 刘蕾. 高光谱图像分类的研究进展[J]. 光学精密工程, 2019, 27(3): 680-693.
	Yan Jing-wen, Chen Hong-da, Liu Lei. Overiew of hyperspectral image classification[J]. Optics and Precision Engineering, 2019, 27(3): 680-693.
5	Zhang H, Li Y, Zhang Y, et al. Spectral-spatial classification of hyperspectral imagery using a dual-channel convolutional neural network[J]. Remote Sensing Letters, 2017, 8(5):438-447.
6	Ouyang N, Zhu T, Lin L P. Convolutional neural network trained by joint loss for hyperspectral image classification[J]. IEEE Geoscience and Remote Sensing Letters, 2018, 16(3): 457-461.
7	Qing Y, Liu W. Hyperspectral image classification based on multi-scale residual network with attention mechanism[J]. Remote Sensing,2021,13(3):No.335.
8	Zhu M H, Jiao L C, Liu F, et al. Residual spectral–spatial attention network for hyperspectral image classification[J]. IEEE Transactions on Geoscience and Remote Sensing, 2020, 59(1): 449-462.
9	欧阳宁, 朱婷, 林乐平. 基于空-谱融合网络的高光谱图像分类方法[J].计算机应用, 2018, 38(7): 1888-1892.
	Ouyang Ning, Zhu Ting, Lin Le-ping. Spatial-spetral fusion network for hyperspectral image classification method[J]. Journal of Computer Applications, 2018, 38(7): 1888-1892.
10	Roy S K, Krishna G, Dubey S R, et al. HybridSN: exploring 3D-2D CNN feature hierarchy for hyperspectral image classification[J]. IEEE Geoscience and Remote Sensing Letters, 2020, 17(2): 277-281.
11	Li Y, Zhang H K, Shen Q. Spectral-spatial classification of hyperspectral imagery with 3D convolutional neural network[J]. Remote Sensing, 2017, 9(1): 67-87.
12	Zhang C, Li G, Du S, et al. Three-dimensional densely connected convolutional network for hyperspectral remote sensing image classification[J]. Journal of Applied Remote Sensing, 2019, 13(1):No.16519.
13	Zhang C J, Li G D, Du S H, et al. Multi-scale dense networks for hyperspectral remote sensing image classification[J]. IEEE Transactions on Geoscience and Remote Sensing, 2019, 57(11): 9201-9222.
14	Wang W, Dou S, Jiang Z, et al. A fast dense spectral-spatial convolution network framework for hyperspectral images classification[J]. Remote Sensing, 2018, 10(7): No.1068.
15	Zhu M H, Jiao L C, Liu F, et al. Residual spectral–spatial attention network for hyperspectral image classification[J]. IEEE Transactions on Geoscience and Remote Sensing, 2020, 59(1): 449-462.
16	Huang G, Chen D, Li T, et al. Multi-scale dense networks for resource efficient image classification[C]//International Conference on Learning Representations, Vancouver, Canada, 2018: 1-14.
17	Chang Y H, Xu J T, Gao Z Y. Multi-scale dense attention network for stereo matching[J]. Electronics, 2020, 9(11):1881-1892.
18	Lu J, Yang J W, Batra D, et al. Hierarchical question-image co-attention for visual question answering[C]//Proceedings of the Advances in Neural Information Processing Systems, Barcelona, Spain, 2016: 289-297.
19	Ma H Y, Li Y J, Ji X P, et al. MsCoa: multi-step co-attention model for multi-label classification[J]. IEEE Access, 2019, 7: 109635-109645.
20	Li M N, Tei K J, Fukazawa Y. An efficient adaptive attention neural network for social recommendation[J]. IEEE Access, 2020, 8: 63595-63606.

相关文章 15

[1]	潘晓英,魏德,赵逸喆. 基于Mask R⁃CNN和上下文卷积神经网络的肺结节检测[J]. 吉林大学学报(工学版), 2022, 52(10): 2419-2427.
[2]	周大可,张超,杨欣. 基于多尺度特征融合及双重注意力机制的自监督三维人脸重建[J]. 吉林大学学报(工学版), 2022, 52(10): 2428-2437.
[3]	赵宏伟,霍东升,王洁,李晓宁. 基于显著性检测的害虫图像分类[J]. 吉林大学学报(工学版), 2021, 51(6): 2174-2181.
[4]	倪涛,刘海强,王林林,邹少元,张红彦,黄玲涛. 基于双向长短期记忆模型的起重机智能操控方法[J]. 吉林大学学报(工学版), 2020, 50(2): 445-453.
[5]	王生生, 郭湑, 张家晨, 王光耀, 赵欣. 基于全局与局部形状特征融合的形状识别算法[J]. 吉林大学学报(工学版), 2016, 46(5): 1627-1632.
[6]	张浩, 刘海明, 吴春国, 张艳梅, 赵天明, 李寿涛. 基于多特征融合的绿色通道车辆检测判定[J]. 吉林大学学报(工学版), 2016, 46(1): 271-276.
[7]	杨欣，刘加，周鹏宇，周大可. 基于多特征融合的粒子滤波自适应目标跟踪算法[J]. 吉林大学学报(工学版), 2015, 45(2): 533-539.
[8]	齐滨, 赵春晖, 王玉磊. 基于支持向量机与相关向量机的高光谱图像分类[J]. 吉林大学学报(工学版), 2013, 43(增刊1): 143-147.
[9]	吴迪, 曹洁. 智能环境下基于核相关权重鉴别分析算法的多特征融合人脸识别[J]. 吉林大学学报(工学版), 2013, 43(02): 439-443.
[10]	王新颖, 刘钢, 谷方明, 肖巍. 三维模型检索中的语义与形状异构特征融合[J]. 吉林大学学报(工学版), 2012, 42(增刊1): 359-363.
[11]	曲智国, 王平, 高颖慧, 王鹏, 沈振康, 李江. 基于同化核分割相同值区域特征融合的边缘检测 [J]. , 2012, (03): 759-765.
[12]	陈绵书, 付平, 李勇, 张慧. 基于范围相似分最小化的图像特征融合[J]. 吉林大学学报(工学版), 2010, 40(增刊): 365-0368.
[13]	王莹, 李文辉. 基于多特征融合的高精度视频火焰检测算法[J]. 吉林大学学报(工学版), 2010, 40(03): 769-0775.
[14]	商飞, 马骏骁, 姚立, 田地, 邱春玲. 基于多特征融合的科学仪器工作状态检测方法[J]. 吉林大学学报(工学版), 2010, 40(02): 545-0548.
[15]	郑雅羽,田翔.陈耀武. 基于时空特征融合的视觉注意模型[J]. 吉林大学学报(工学版), 2009, 39(06): 1625-1630.

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed

方法	3D-CNN	3D-DenseNet	MSDN（n=4）	HSFF_C	HSFF
1	97.23	97.31	99.10	97.29	100
2	98.95	96.51	96.73	98.32	99.59
3	96.87	93.88	99.85	99.25	99.49
4	95.77	94.95	94.88	97.84	98.90
5	90.68	98.97	94.13	99.64	99.55
6	92.72	97.72	98.95	99.55	99.96
7	94.68	92.19	98.20	99.52	99.47
8	99.82	98.21	95.81	99.69	100
9	84.63	93.01	99.70	98.33	100
10	98.01	96.20	93.80	99.13	99.08
11	98.81	97.41	96.05	99.60	99.70
12	95.41	95.03	96.94	92.67	99.59
13	99.35	99.86	95.74	99.79	100
14	99.19	99.19	99.39	99.16	99.83
15	93.09	95.52	95.50	97.43	98.90
16	96.08	93.43	95.88	89.19	96.12
OA/%	96.60±2.58	96.92±0.40	97.06±0.60	98.70±0.41	99.57±0.11
AA/%	95.71±1.61	96.21±1.24	96.87±1.13	97.90±0.35	99.38±0.17
Kappa×100	96.13±2.92	96.49±0.46	96.64±0.69	98.52±0.47	99.50±0.12

方法	3D?CNN	3D?DenseNet	MSDN（n=4）	HSFF_C	HSFF
1	98.63	99.65	99.87	99.96	99.81
2	98.49	91.26	99.48	99.33	99.65
3	92.05	91.37	91.98	93.55	98.90
4	89.65	92.53	90.96	95.51	95.95
5	62.34	91.06	87.78	94.14	93.36
6	96.89	93.42	95.94	98.64	99.32
7	97.56	95.05	93.04	97.56	99.49
8	97.31	96.74	95.86	98.81	99.77
9	99.81	95.24	94.66	99.97	100
10	99.44	96.74	99.40	100	100
11	100	99.47	99.96	100	99.86
12	99.62	98.15	99.08	99.31	100
13	100	99.91	100	100	100
OA/%	96.19±1.54	96.23±1.99	97.36±0.43	98.92±0.28	99.41±0.17
AA/%	94.75±1.57	95.43±1.73	96.00±0.58	98.21±0.50	98.93±0.39
Kappa×100	95.76±1.71	95.80±2.22	97.07±0.47	98.80±0.31	99.35±0.19

方法	3D?CNN	3D?DenseNet	MSDN（n=4）	HSFF_C	HSFF
1	93.33	98.42	98.33	99.66	99.87
2	99.22	99.88	99.94	99.94	99.96
3	96.15	96.79	97.59	99.28	99.63
4	99.84	99.78	99.88	99.90	99.91
5	100	99.38	99.93	99.98	100
6	94.15	99.76	99.91	99.56	99.98
7	99.93	97.59	99.12	99.87	99.98
8	87.85	97.17	96.76	98.45	99.35
9	99.93	99.62	99.96	100	99.95
OA/%	96.33±1.99	99.15±0.15	99.26±0.07	99.69±0.06	99.87±0.04
AA/%	96.67±2.03	98.71±0.25	99.04±0.14	99.62±0.09	99.84±0.04
Kappa×100	95.12±2.66	98.87±0.20	99.02±0.10	99.58±0.09	99.83±0.06

方法	3D?CNN	3D?DenseNet	MSDN（n=4）	HSFF_C	HSFF
1	99.97	98.83	99.81	99.91	99.99
2	99.24	100	99.80	100	100
3	99.86	99.91	99.79	99.77	100
4	99.00	99.71	99.70	99.84	99.88
5	99.71	99.95	99.83	99.64	99.99
6	99.98	99.97	99.76	99.89	99.99
7	71.35	99.99	100	100	100
8	99.94	99.45	99.66	99.89	99.92
9	98.40	99.99	99.78	99.43	100
10	99.73	99.63	99.51	99.55	99.94
11	99.71	99.30	99.66	99.72	99.98
12	99.97	99.98	100	99.67	99.96
13	99.54	99.62	100	99.79	99.93
14	96.37	99.75	98.03	99.51	100
15	91.09	98.42	99.82	99.80	99.82
16	97.75	100	99.48	100	100
OA/%	89.94±2.14	99.56±0.10	99.68±0.11	99.72±0.06	99.95±0.03
AA/%	96.98±0.69	99.66±0.13	99.66±0.12	99.70±0.10	99.96±0.02
Kappa×100	88.67±2.44	99.51±0.11	99.71±0.09	99.77±0.07	99.94±0.03

数据	t/s		3D?DenseNet	MSDN（n=4）	HSFF_C	HSFF
IP	训练	1309.3	6136.7	9106.2	2122.7	3898.3
IP	测试	5.5	26.8	40.7	11.8	19.8
UP	训练	1694.1	7394.6	6796.1	3879.1	5526.9
UP	测试	14.7	60.9	61.9	39.6	56.9
KSC	训练	619.2	3205.1	2008.5	1028.3	1809.6
KSC	测试	2.5	12.8	9.1	5.7	7.1
Epochs		300	200	200	300	300