多特征融合和孪生注意力网络的高分辨率遥感图像目标检测

doi:10.13229/j.cnki.jdxbgxb.20221286

摘要/Abstract

摘要：

为提高高分辨率遥感图像目标检测效果，本文将多特征融合方法和孪生注意力网络相结合，提出一种新的目标检测方法。构建遥感图像目标检测的整体框架，基于锚框模型对遥感图像目标进行多层特征的提取及融合；运用孪生注意力网络对遥感图像目标实时视觉跟踪检测，引入通道和空间的双重自注意力机制，提高目标图像的特征表达能力，由此得到更加精准的检测结果。实验分析结果表明，本文方法的平均总体精度为93.8，F₁指数平均值为0.88，Kappa系数平均值为0.93，均明显高于对比方法，说明本文方法具有较好的检测效果。

关键词: 多特征融合, 高分辨率, 遥感图像, 孪生注意力, 目标检测, 语义特征

Abstract:

In order to improve the effect of object detection in high-resolution remote sensing images， this paper proposes a new object detection method by combining multi-feature fusion method and twin attention network. The overall framework of remote sensing image target detection is constructed， and the multi-layer features of remote sensing image target are extracted and fused based on the anchor frame model. The twin attention network is used for real-time visual tracking and detection of remote sensing image targets， and the dual self-attention mechanism of channel and space is introduced to improve the feature expression ability of target images， so as to get more accurate detection results. Through the analysis of experiments， the average overall accuracy of the proposed method is 93.8， the average F₁ index is 0.88， and the average Kappa coefficient is 0.93， which are significantly higher than the comparison method， indicating that the proposed method has a good detection effect.

Key words: multi-feature fusion, high resolution, remote sensing image, twin attention, object detection, semantic features

中图分类号:

TP751

王春华,李恩泽,肖敏. 多特征融合和孪生注意力网络的高分辨率遥感图像目标检测[J]. 吉林大学学报(工学版), 2024, 54(1): 240-250.

Chun-hua WANG,En-ze LI,Min XIAO. Object detection in high-resolution remote sensing images based on multi-feature fusion and twin attention network[J]. Journal of Jilin University(Engineering and Technology Edition), 2024, 54(1): 240-250.

图/表 11

图1

图2

图3

表1

图4

图5

图6

图7

图8

图9

表2

参考文献 22

1	李雄飞, 吴佳婧, 张小利, 等.基于相对总变差结构提取的遥感图像融合[J]. 吉林大学学报: 工学版,2021, 51(5): 1775-1784.
	Li Xiong-fei, Wu Jia-jing, Zhang Xiao-li, et al. Remote sensing image fusion based on relative total variation structure extraction[J]. Journal of Jilin University (Engineering and Technology Edition), 2021,51(5): 1775-1784.
2	朱节中, 陈永, 柯福阳,等. 基于Siam-UNet++的高分辨率遥感影像建筑物变化检测[J]. 计算机应用研究, 2021, 38(11): 3460-3465.
	Zhu Jie-zhong, Chen Yong, Ke Fu-yang, et al. Building change detection from high resolution remote sensing imagery based on Siam-UNet++[J]. Application Research of Computers, 2021, 38(11): 3460-3465.
3	张筱晗, 姚力波, 吕亚飞, 等. 双向特征融合的数据自适应SAR图像舰船目标检测模型[J]. 中国图像图形学报, 2020, 25(9): 1943-1952.
	Zhang Xiao-han, Yao Li-bo, Lv Ya-fei, et al. Data-adaptive single-shot ship detector with a bidirectional feature fusion module for SAR images[J]. Journal of Image and Graphics, 2020, 25(9): 1943-1952.
4	杨耘, 李龙威, 高思岩,等. 基于YOLOv3网络训练优化的高分辨率遥感影像目标检测[J]. 激光与光电子学进展, 2021, 58(16): 147-153.
	Yang Yun, Li Long-wei, Gao Si-yan, et al. Objects detection from high-resolution remote sensing imagery using training-optimized YOLOv3 network[J]. Laser & Optoelectronics Progress, 2021, 58(16): 147-153.
5	宋忠浩, 谷雨, 陈旭,等. 基于加权策略的高分辨率遥感图像目标检测[J]. 计算机工程与应用, 2021, 57(13): 199-206.
	Song Zhong-hao, Gu Yu, Chen Xu, et al. Target detection in high-resolution remote sensing image based on weighted strategy[J]. Computer Engineering and Applications, 2021, 57(13): 199-206.
6	廖育荣, 王海宁, 林存宝, 等. 基于深度学习的光学遥感图像目标检测研究进展[J].通信学报, 2022, 43(5): 190-203.
	Liao Yu-rong, Wang Hai-ning, Lin Cun-bao, et al. Research progress of object detection in optical remote sensing images based on depth learning[J]. Journal of Communications, 2022,43(5): 190-203.
7	陈雪云, 黄金汉, 胡子灿, 等. 基于线性逻辑矢量模式的遥感图像目标检测[J]. 浙江大学学报: 工学版, 2022, 56(1): 47-55.
	Chen Xue-yun, Huang Jin-han, Hu Zi-can, et al. Remote sensing image target detection based on linear logic vector mode[J]. Journal of Zhejiang University (Engineering Science), 2022, 56 (1): 47-55.
8	汪西莉, 梁敏, 刘涛. 特征增强的单阶段遥感图像目标检测模型[J]. 西安电子科技大学学报, 2022, 49(3): 160-170.
	Wang Xi-li, Liang Min, Liu Tao. Single stage target detection model of remote sensing image with feature enhancement [J]. Journal of Xidian University, 2022,49 (3): 160-170.
9	袁洲, 郭海涛, 卢俊, 等. 融合UNet++网络和注意力机制的高分辨率遥感影像变化检测算法[J]. 测绘科学技术学报, 2021, 38(2): 155-159.
	Yuan Zhou, Guo Hai-tao, Lu Jun, et al. High-resolution remote sensing image change detection technology based on UNet++and attention mechanism[J]. Journal of Geomatics Science and Technology, 2021, 38(2): 155-159.
10	郑哲, 雷琳, 孙浩, 等. FAGNet: 基于MAFPN和GVR的遥感图像多尺度目标检测算法[J]. 计算机辅助设计与图形学学报, 2021, 33(6): 883-894.
	Zheng Zhe, Lei Lin, Sun Hao, et al. FAGNet: multi-scale object detection method in remote sensing images by combining MAFPN and GVR[J]. Journal of Computer-Aided Design & Computer Graphics, 2021, 33(6): 883-894.
11	赵琰, 赵凌君, 匡纲要. 基于注意力机制特征融合网络的SAR图像飞机目标快速检测[J]. 电子学报, 2021, 49(9): 1665-1674.
	Zhao Yan, Zhao Ling-jun, Kuang Gang-yao. Attention feature fusion network for rapid aircraft detection in SAR images[J]. Acta Electronica Sinica, 2021, 49(9): 1665-1674.
12	严继伟, 苏娟, 李义红. 基于Ghost卷积与注意力机制的SAR图像建筑物检测算法[J]. 兵工学报, 2022, 43(7): 1667-1675.
	Yan Ji-wei, Su Juan, Li Yi-hong. Building detection algorithm in SAR images based on ghost convolution and attention mechanisms[J]. Acta Armamentarii, 2022, 43(7): 1667-1675.
13	王玲, 王家沛, 王鹏, 等. 融合注意力机制的孪生网络目标跟踪算法研究[J]. 计算机工程与应用, 2021, 57(8): 169-174.
	Wang Ling, Wang Jia-pei, Wang Peng, et al. Siamese network tracking algorithms for hierarchical fusion of attention mechanism[J]. Computer Engineering and Applications, 2021, 57(8): 169-174.
14	张福玲, 张少敏, 支力佳, 等. 融合注意力机制和特征金字塔网络的CT图像肺结节检测[J]. 中国图像图形学报, 2021, 26(9): 2156-2170.
	Zhang Fu-ling, Zhang Shao-min, Zhi Li-jia, et al. Detection of pulmonary nodules in CT images by combining an attention mechanism and a feature pyramid network[J]. Journal of Image and Graphics, 2021, 26(9): 2156-2170.
15	兰旭婷, 郭中华, 李昌昊. 基于注意力与特征融合的光学遥感图像飞机目标检测[J]. 液晶与显示, 2021, 36(11):1506-1515.
	Lan Xu-ting, Guo Zhong-hua, Li Chang-hao. Attention and feature fusion for aircraft target detection in optical remote sensing images[J]. Chinese Journal of Liquid Crystals and Displays, 2021, 36(11):1506-1515.
16	石瑞姣, 陈后金, 李居朋,等. 基于注意力和多级特征融合的铁路场景小尺度行人检测算法[J]. 铁道学报, 2022, 44(5):76-83.
	Shi Rui-jiao, Chen Hou-jin, Li Ju-peng, et al. Small-scale pedestrian detection algorithm based on attention and multi-level feature fusion for railway[J]. Journal of the China Railway Society, 2022, 44(5):76-83.
17	魏恺轩, 付莹. 基于重参数化多尺度融合网络的高效极暗光原始图像降噪[J]. 计算机科学, 2022, 49(8): 120-126.
	Wei Kai-xuan, Fu Ying. Re-parameterized multi-scale fusion network for efficient extreme low-light raw denoising[J]. Computer Science, 2022, 49(8):120-126.
18	席志红, 袁昆鹏. 基于残差通道注意力和多级特征融合的图像超分辨率重建[J]. 激光与光电子学进展, 2020, 57(4):262-270.
	Xi Zhi-hong, Yuan Kun-peng. Super-resolution image reconstruction based on residual channel attention and multilevel feature fusion[J]. Laser & Optoelectronics Progress, 2020, 57(4):262-270.
19	沙苗苗,李宇,李安. 改进Faster R-CNN的遥感图像多尺度飞机目标检测[J]. 遥感学报,2022,26(8):1624-1635.
	Sha Miao-miao, Li Yu, Li An. Improved faster R-CNN multi-scale aircraft target detection from remote sensing images[J]. National Remote Sensing Bulletin, 2022,26 (8): 1624-1635.
20	李雪,李晓艳,王鹏,等. 结合双注意力与特征融合的孪生网络目标跟踪[J]. 北京邮电大学学报,2022,45(4):116-122.
	Li Xue, Li Xiao-yan, Wang Peng, et al. Twin network target tracking combined with dual attention and feature fusion[J]. Journal of Beijing University of Posts and Telecommunications, 2022,45(4): 116-122.
21	张志远, 杨帆. 结合多注意力机制的自监督目标跟踪[J]. 计算机工程与设计, 2021, 42(12):3502-3509.
	Zhang Zhi-Yuan, Yang Fan. Self-supervised object tracking based on multi-attention mechanism[J]. Computer Engineering and Design, 2021, 42(12):3502-3509.
22	李卫华,李小春,全卫澎. 多尺度多特征融合的高分辨率遥感影像变化检测[J]. 遥感科学,2017,5(1):52-57.
	Li Wei-Hua, Li Xiao-Chun, Quan Wei-Peng. Change detection of high resolution remote sensing images on multi-scale and multi-feature fusion[J]. Remote Sensing Science,2017,5(1):52-57.

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed

数据集	图像规模	图像尺寸	图像类别
VEDAI	3 023	800	1
TAS	550	1 000	15
DOTA	2 122	1 080	20
NWPU VHR	6 552	1 280	9
UCAS-AOD	851	1 024	3
DLR 3k Vehicle	4 110	700~1 000	3
HRSC2016	245	1 000	2
DIOR	55	5 326	4
SZTAKI-INRIA	5410	1 000	1
RSOD	28 532	800	10

数据集及平均值	总体精度/%			F₁指数			Kappa系数
数据集及平均值	本文方法	YOLOv3网络训练方法	方法加权融合	本文方法	YOLOv3网络训练方法	加权融合方法	本文方法	YOLOv3网络训练方法	加权融合方法
VEDAI	96.3	83.6	85.2	0.83	0.76	0.72	0.93	0.86	0.75
TAS	91.7	87.5	85.5	0.80	0.72	0.75	0.90	0.82	0.81
DOTA	95.2	88.8	84.8	0.91	0.74	0.75	0.94	0.83	0.76
NWPU VHR	95.5	85.8	88.5	0.96	0.69	0.73	0.95	0.72	0.83
UCAS-AOD	92.2	85.1	85.4	0.83	0.71	0.64	0.96	0.82	0.81
DLR 3k Vehicle	92.4	85.4	88.5	0.95	0.78	0.82	0.91	0.85	0.82
HRSC2016	95. 4	85. 5	81.2	0.80	0.68	0.76	0.92	0.75	0.85
DIOR	94.5	84.6	81.5	0.85	0.74	0.64	0.91	0.72	0.78
SZTAKI-INRIA	91.2	85.8	85.3	0.98	0.79	0.65	0.95	0.88	0.75
RSOD	91.2	84.5	85.4	0.85	0.65	0.69	0.95	0.85	0.88
平均值	92.30	84.97	84.07	0.89	0.73	0.66	0.94	0.82	0.80

[1]	蔡志丹,方明,李喆,许佳路. 基于高斯曲率和加权图总变分正则化的遥感图像盲去模糊算法[J]. 吉林大学学报(工学版), 2023, 53(9): 2649-2658.
[2]	郭晓新,李佳慧,张宝亮. 基于高分辨率网络的视杯和视盘的联合分割[J]. 吉林大学学报(工学版), 2023, 53(8): 2350-2357.
[3]	陶博,颜伏伍,尹智帅,武冬梅. 基于高精度地图增强的三维目标检测算法[J]. 吉林大学学报(工学版), 2023, 53(3): 802-809.
[4]	薛珊,张亚亮,吕琼莹,曹国华. 复杂背景下的反无人机系统目标检测算法[J]. 吉林大学学报(工学版), 2023, 53(3): 891-901.
[5]	成丽波,李新月,李喆,贾小宁. 基于曲波变换与拟合优度检验的遥感图像去噪方法[J]. 吉林大学学报(工学版), 2023, 53(11): 3207-3213.
[6]	马为駽,张䶮,马传香,朱飒. 不同光照条件下含噪遥感图像边缘检测算法[J]. 吉林大学学报(工学版), 2023, 53(1): 241-247.
[7]	朱冰,李紫薇,李奇. 基于改进SegNet的遥感图像建筑物分割方法[J]. 吉林大学学报(工学版), 2023, 53(1): 248-254.
[8]	高明华,杨璨. 基于改进卷积神经网络的交通目标检测方法[J]. 吉林大学学报(工学版), 2022, 52(6): 1353-1361.
[9]	曲优,李文辉. 基于锚框变换的单阶段旋转目标检测方法[J]. 吉林大学学报(工学版), 2022, 52(1): 162-173.
[10]	赵宏伟,霍东升,王洁,李晓宁. 基于显著性检测的害虫图像分类[J]. 吉林大学学报(工学版), 2021, 51(6): 2174-2181.
[11]	李雄飞,吴佳婧,张小利,王泽宇,冯云丛. 基于相对总变差结构提取的遥感图像融合[J]. 吉林大学学报(工学版), 2021, 51(5): 1775-1784.
[12]	潘德伦,冀隽,张跃进. 基于运动矢量空间编码的视频监控动态目标检测方法[J]. 吉林大学学报(工学版), 2021, 51(4): 1370-1374.
[13]	金立生,郭柏苍,王芳荣,石健. 基于改进YOLOv3的车辆前方动态多目标检测算法[J]. 吉林大学学报(工学版), 2021, 51(4): 1427-1436.
[14]	陈雪云,许韬,黄小巧. 基于条件生成对抗网络的医学细胞图像生成检测方法[J]. 吉林大学学报(工学版), 2021, 51(4): 1414-1419.
[15]	兰凤崇,李继文,陈吉清. 面向动态场景复合深度学习与并行计算的DG-SLAM算法[J]. 吉林大学学报(工学版), 2021, 51(4): 1437-1446.