多特征融合和孪生注意力网络的高分辨率遥感图像目标检测

doi:10.13229/j.cnki.jdxbgxb.20221286

Abstract

Abstract:

In order to improve the effect of object detection in high-resolution remote sensing images， this paper proposes a new object detection method by combining multi-feature fusion method and twin attention network. The overall framework of remote sensing image target detection is constructed， and the multi-layer features of remote sensing image target are extracted and fused based on the anchor frame model. The twin attention network is used for real-time visual tracking and detection of remote sensing image targets， and the dual self-attention mechanism of channel and space is introduced to improve the feature expression ability of target images， so as to get more accurate detection results. Through the analysis of experiments， the average overall accuracy of the proposed method is 93.8， the average F₁ index is 0.88， and the average Kappa coefficient is 0.93， which are significantly higher than the comparison method， indicating that the proposed method has a good detection effect.

Key words: multi-feature fusion, high resolution, remote sensing image, twin attention, object detection, semantic features

CLC Number:

TP751

Chun-hua WANG,En-ze LI,Min XIAO. Object detection in high-resolution remote sensing images based on multi-feature fusion and twin attention network[J].Journal of Jilin University(Engineering and Technology Edition), 2024, 54(1): 240-250.

Figures/Tables 11

Fig.1

Fig.2

Fig.3

Table 1

Fig.4

Fig.5

Fig.6

Fig.7

Fig.8

Fig.9

Table 2

References 22

1	李雄飞, 吴佳婧, 张小利, 等.基于相对总变差结构提取的遥感图像融合[J]. 吉林大学学报: 工学版,2021, 51(5): 1775-1784.
	Li Xiong-fei, Wu Jia-jing, Zhang Xiao-li, et al. Remote sensing image fusion based on relative total variation structure extraction[J]. Journal of Jilin University (Engineering and Technology Edition), 2021,51(5): 1775-1784.
2	朱节中, 陈永, 柯福阳,等. 基于Siam-UNet++的高分辨率遥感影像建筑物变化检测[J]. 计算机应用研究, 2021, 38(11): 3460-3465.
	Zhu Jie-zhong, Chen Yong, Ke Fu-yang, et al. Building change detection from high resolution remote sensing imagery based on Siam-UNet++[J]. Application Research of Computers, 2021, 38(11): 3460-3465.
3	张筱晗, 姚力波, 吕亚飞, 等. 双向特征融合的数据自适应SAR图像舰船目标检测模型[J]. 中国图像图形学报, 2020, 25(9): 1943-1952.
	Zhang Xiao-han, Yao Li-bo, Lv Ya-fei, et al. Data-adaptive single-shot ship detector with a bidirectional feature fusion module for SAR images[J]. Journal of Image and Graphics, 2020, 25(9): 1943-1952.
4	杨耘, 李龙威, 高思岩,等. 基于YOLOv3网络训练优化的高分辨率遥感影像目标检测[J]. 激光与光电子学进展, 2021, 58(16): 147-153.
	Yang Yun, Li Long-wei, Gao Si-yan, et al. Objects detection from high-resolution remote sensing imagery using training-optimized YOLOv3 network[J]. Laser & Optoelectronics Progress, 2021, 58(16): 147-153.
5	宋忠浩, 谷雨, 陈旭,等. 基于加权策略的高分辨率遥感图像目标检测[J]. 计算机工程与应用, 2021, 57(13): 199-206.
	Song Zhong-hao, Gu Yu, Chen Xu, et al. Target detection in high-resolution remote sensing image based on weighted strategy[J]. Computer Engineering and Applications, 2021, 57(13): 199-206.
6	廖育荣, 王海宁, 林存宝, 等. 基于深度学习的光学遥感图像目标检测研究进展[J].通信学报, 2022, 43(5): 190-203.
	Liao Yu-rong, Wang Hai-ning, Lin Cun-bao, et al. Research progress of object detection in optical remote sensing images based on depth learning[J]. Journal of Communications, 2022,43(5): 190-203.
7	陈雪云, 黄金汉, 胡子灿, 等. 基于线性逻辑矢量模式的遥感图像目标检测[J]. 浙江大学学报: 工学版, 2022, 56(1): 47-55.
	Chen Xue-yun, Huang Jin-han, Hu Zi-can, et al. Remote sensing image target detection based on linear logic vector mode[J]. Journal of Zhejiang University (Engineering Science), 2022, 56 (1): 47-55.
8	汪西莉, 梁敏, 刘涛. 特征增强的单阶段遥感图像目标检测模型[J]. 西安电子科技大学学报, 2022, 49(3): 160-170.
	Wang Xi-li, Liang Min, Liu Tao. Single stage target detection model of remote sensing image with feature enhancement [J]. Journal of Xidian University, 2022,49 (3): 160-170.
9	袁洲, 郭海涛, 卢俊, 等. 融合UNet++网络和注意力机制的高分辨率遥感影像变化检测算法[J]. 测绘科学技术学报, 2021, 38(2): 155-159.
	Yuan Zhou, Guo Hai-tao, Lu Jun, et al. High-resolution remote sensing image change detection technology based on UNet++and attention mechanism[J]. Journal of Geomatics Science and Technology, 2021, 38(2): 155-159.
10	郑哲, 雷琳, 孙浩, 等. FAGNet: 基于MAFPN和GVR的遥感图像多尺度目标检测算法[J]. 计算机辅助设计与图形学学报, 2021, 33(6): 883-894.
	Zheng Zhe, Lei Lin, Sun Hao, et al. FAGNet: multi-scale object detection method in remote sensing images by combining MAFPN and GVR[J]. Journal of Computer-Aided Design & Computer Graphics, 2021, 33(6): 883-894.
11	赵琰, 赵凌君, 匡纲要. 基于注意力机制特征融合网络的SAR图像飞机目标快速检测[J]. 电子学报, 2021, 49(9): 1665-1674.
	Zhao Yan, Zhao Ling-jun, Kuang Gang-yao. Attention feature fusion network for rapid aircraft detection in SAR images[J]. Acta Electronica Sinica, 2021, 49(9): 1665-1674.
12	严继伟, 苏娟, 李义红. 基于Ghost卷积与注意力机制的SAR图像建筑物检测算法[J]. 兵工学报, 2022, 43(7): 1667-1675.
	Yan Ji-wei, Su Juan, Li Yi-hong. Building detection algorithm in SAR images based on ghost convolution and attention mechanisms[J]. Acta Armamentarii, 2022, 43(7): 1667-1675.
13	王玲, 王家沛, 王鹏, 等. 融合注意力机制的孪生网络目标跟踪算法研究[J]. 计算机工程与应用, 2021, 57(8): 169-174.
	Wang Ling, Wang Jia-pei, Wang Peng, et al. Siamese network tracking algorithms for hierarchical fusion of attention mechanism[J]. Computer Engineering and Applications, 2021, 57(8): 169-174.
14	张福玲, 张少敏, 支力佳, 等. 融合注意力机制和特征金字塔网络的CT图像肺结节检测[J]. 中国图像图形学报, 2021, 26(9): 2156-2170.
	Zhang Fu-ling, Zhang Shao-min, Zhi Li-jia, et al. Detection of pulmonary nodules in CT images by combining an attention mechanism and a feature pyramid network[J]. Journal of Image and Graphics, 2021, 26(9): 2156-2170.
15	兰旭婷, 郭中华, 李昌昊. 基于注意力与特征融合的光学遥感图像飞机目标检测[J]. 液晶与显示, 2021, 36(11):1506-1515.
	Lan Xu-ting, Guo Zhong-hua, Li Chang-hao. Attention and feature fusion for aircraft target detection in optical remote sensing images[J]. Chinese Journal of Liquid Crystals and Displays, 2021, 36(11):1506-1515.
16	石瑞姣, 陈后金, 李居朋,等. 基于注意力和多级特征融合的铁路场景小尺度行人检测算法[J]. 铁道学报, 2022, 44(5):76-83.
	Shi Rui-jiao, Chen Hou-jin, Li Ju-peng, et al. Small-scale pedestrian detection algorithm based on attention and multi-level feature fusion for railway[J]. Journal of the China Railway Society, 2022, 44(5):76-83.
17	魏恺轩, 付莹. 基于重参数化多尺度融合网络的高效极暗光原始图像降噪[J]. 计算机科学, 2022, 49(8): 120-126.
	Wei Kai-xuan, Fu Ying. Re-parameterized multi-scale fusion network for efficient extreme low-light raw denoising[J]. Computer Science, 2022, 49(8):120-126.
18	席志红, 袁昆鹏. 基于残差通道注意力和多级特征融合的图像超分辨率重建[J]. 激光与光电子学进展, 2020, 57(4):262-270.
	Xi Zhi-hong, Yuan Kun-peng. Super-resolution image reconstruction based on residual channel attention and multilevel feature fusion[J]. Laser & Optoelectronics Progress, 2020, 57(4):262-270.
19	沙苗苗,李宇,李安. 改进Faster R-CNN的遥感图像多尺度飞机目标检测[J]. 遥感学报,2022,26(8):1624-1635.
	Sha Miao-miao, Li Yu, Li An. Improved faster R-CNN multi-scale aircraft target detection from remote sensing images[J]. National Remote Sensing Bulletin, 2022,26 (8): 1624-1635.
20	李雪,李晓艳,王鹏,等. 结合双注意力与特征融合的孪生网络目标跟踪[J]. 北京邮电大学学报,2022,45(4):116-122.
	Li Xue, Li Xiao-yan, Wang Peng, et al. Twin network target tracking combined with dual attention and feature fusion[J]. Journal of Beijing University of Posts and Telecommunications, 2022,45(4): 116-122.
21	张志远, 杨帆. 结合多注意力机制的自监督目标跟踪[J]. 计算机工程与设计, 2021, 42(12):3502-3509.
	Zhang Zhi-Yuan, Yang Fan. Self-supervised object tracking based on multi-attention mechanism[J]. Computer Engineering and Design, 2021, 42(12):3502-3509.
22	李卫华,李小春,全卫澎. 多尺度多特征融合的高分辨率遥感影像变化检测[J]. 遥感科学,2017,5(1):52-57.
	Li Wei-Hua, Li Xiao-Chun, Quan Wei-Peng. Change detection of high resolution remote sensing images on multi-scale and multi-feature fusion[J]. Remote Sensing Science,2017,5(1):52-57.

Related Articles 15

[1]	Zhi-dan CAI,Ming FANG,Zhe LI,Jia-lu XU. Blind remote sensing image deblurring algorithm based on Gaussian curvature and reweighted graph total variation [J]. Journal of Jilin University(Engineering and Technology Edition), 2023, 53(9): 2649-2658.
[2]	Bo TAO,Fu-wu YAN,Zhi-shuai YIN,Dong-mei WU. 3D object detection based on high⁃precision map enhancement [J]. Journal of Jilin University(Engineering and Technology Edition), 2023, 53(3): 802-809.
[3]	Shan XUE,Ya-liang ZHANG,Qiong-ying LYU,Guo-hua CAO. Anti⁃unmanned aerial vehicle system object detection algorithm under complex background [J]. Journal of Jilin University(Engineering and Technology Edition), 2023, 53(3): 891-901.
[4]	Li-bo CHENG,Xin-yue LI,Zhe LI,Xiao-ning JIA. Remote sensing image denoising method based on curvelet transform and goodness-of-fit test [J]. Journal of Jilin University(Engineering and Technology Edition), 2023, 53(11): 3207-3213.
[5]	Wei-xuan MA,Yan ZHANG,Chuan-xiang MA,Sa ZHU. Edge detection algorithm of noisy remote sensing image under different illumination conditions [J]. Journal of Jilin University(Engineering and Technology Edition), 2023, 53(1): 241-247.
[6]	Bing ZHU,Zi-wei LI,Qi LI. Building segmentation method of remote sensing image based on improved SegNet [J]. Journal of Jilin University(Engineering and Technology Edition), 2023, 53(1): 248-254.
[7]	Xiang-jiu CHE,He-yuan CHEN. Muti⁃Object dishes detection algorithm based on improved YOLOv4 [J]. Journal of Jilin University(Engineering and Technology Edition), 2022, 52(11): 2662-2668.
[8]	You QU,Wen-hui LI. Single-stage rotated object detection network based on anchor transformation [J]. Journal of Jilin University(Engineering and Technology Edition), 2022, 52(1): 162-173.
[9]	Hong-wei ZHAO,Dong-sheng HUO,Jie WANG,Xiao-ning LI. Image classification of insect pests based on saliency detection [J]. Journal of Jilin University(Engineering and Technology Edition), 2021, 51(6): 2174-2181.
[10]	Xiong-fei LI,Jia-jing WU,Xiao-li ZHANG,Ze-yu WANG,Yun-cong FENG. Remote sensing image fusion algorithm based on relative total variation structure extraction [J]. Journal of Jilin University(Engineering and Technology Edition), 2021, 51(5): 1775-1784.
[11]	Li-sheng JIN,Bai-cang GUO,Fang-rong WANG,Jian SHI. Dynamic multiple object detection algorithm for vehicle forward based on improved YOLOv3 [J]. Journal of Jilin University(Engineering and Technology Edition), 2021, 51(4): 1427-1436.
[12]	Feng-chong LAN,Ji-wen LI,Ji-qing CHEN. DG-SLAM algorithm for dynamic scene compound deep learning and parallel computing [J]. Journal of Jilin University(Engineering and Technology Edition), 2021, 51(4): 1437-1446.
[13]	Zhe-ming YUAN,Hong-jie YUAN,Yu-xuan YAN,Qian LI,Shuang-qing LIU,Si-qiao TAN. Automatic recognition and classification of field insects based on lightweight deep learning model [J]. Journal of Jilin University(Engineering and Technology Edition), 2021, 51(3): 1131-1139.
[14]	Tao XU,Ke MA,Cai-hua LIU. Multi object pedestrian tracking based on deep learning [J]. Journal of Jilin University(Engineering and Technology Edition), 2021, 51(1): 27-38.
[15]	Chang-fu ZONG,Long WEN,Lei HE. Object detection based on Euclidean clustering algorithm with 3D laser scanner [J]. Journal of Jilin University(Engineering and Technology Edition), 2020, 50(1): 107-113.

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed

Comments

Recommended 0

No Suggested Reading articles found!

数据集	图像规模	图像尺寸	图像类别
VEDAI	3 023	800	1
TAS	550	1 000	15
DOTA	2 122	1 080	20
NWPU VHR	6 552	1 280	9
UCAS-AOD	851	1 024	3
DLR 3k Vehicle	4 110	700~1 000	3
HRSC2016	245	1 000	2
DIOR	55	5 326	4
SZTAKI-INRIA	5410	1 000	1
RSOD	28 532	800	10

数据集及平均值	总体精度/%			F₁指数			Kappa系数
数据集及平均值	本文方法	YOLOv3网络训练方法	方法加权融合	本文方法	YOLOv3网络训练方法	加权融合方法	本文方法	YOLOv3网络训练方法	加权融合方法
VEDAI	96.3	83.6	85.2	0.83	0.76	0.72	0.93	0.86	0.75
TAS	91.7	87.5	85.5	0.80	0.72	0.75	0.90	0.82	0.81
DOTA	95.2	88.8	84.8	0.91	0.74	0.75	0.94	0.83	0.76
NWPU VHR	95.5	85.8	88.5	0.96	0.69	0.73	0.95	0.72	0.83
UCAS-AOD	92.2	85.1	85.4	0.83	0.71	0.64	0.96	0.82	0.81
DLR 3k Vehicle	92.4	85.4	88.5	0.95	0.78	0.82	0.91	0.85	0.82
HRSC2016	95. 4	85. 5	81.2	0.80	0.68	0.76	0.92	0.75	0.85
DIOR	94.5	84.6	81.5	0.85	0.74	0.64	0.91	0.72	0.78
SZTAKI-INRIA	91.2	85.8	85.3	0.98	0.79	0.65	0.95	0.88	0.75
RSOD	91.2	84.5	85.4	0.85	0.65	0.69	0.95	0.85	0.88
平均值	92.30	84.97	84.07	0.89	0.73	0.66	0.94	0.82	0.80

Object detection in high-resolution remote sensing images based on multi-feature fusion and twin attention network

RICH HTML

PDF (PC)