基于高精度地图增强的三维目标检测算法

doi:10.13229/j.cnki.jdxbgxb20221254

Abstract

Abstract:

A novel 3D object detection algorithm based on high-precision map enhancement （HME3D） was proposed by integrating high-precision map information into the backbone detection network. Specifically， the high-precision map feature extraction module （HFE） was constructed by combining traditional convolution and transformer to achieve efficient extraction of map features. In addition， the auxiliary supervision network （MEES） based on map edge enhancement was designed to improve the performance of the main 3D object detection task. Finally， the advantages of the proposed model in this paper are verified on the challenging nuScenes dataset， which improves the accuracy of the LiDAR baseline model by 2.81 mAP.

Key words: computer application, autonomous driving, environment perception, 3D object detection, high-precision map

CLC Number:

TP391

Bo TAO,Fu-wu YAN,Zhi-shuai YIN,Dong-mei WU. 3D object detection based on high⁃precision map enhancement[J].Journal of Jilin University(Engineering and Technology Edition), 2023, 53(3): 802-809.

Figures/Tables 9

Fig.1

Fig.2

Fig.3

Fig.4

Table 1

Table 2

Table 3

Table 4

Fig.5

References 35

1	Levinson J, Askeland J, Becker J, et al. Towards fully autonomous driving: systems and algorithms[C]∥2011 IEEE Intelligent Vehicles Symposium (IV), Baden-Baden, Germany, 2011: 163-168.
2	Tan M, Pang R, Le Q V. Efficientdet: scalable and efficient object detection[C]∥Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA, 2020: 10781-10790.
3	Lin T Y, Dollár P, Girshick R, et al. Feature pyramid networks for object detection[C]∥Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA, 2017: 2117-2125.
4	Zhao Q, Sheng T, Wang Y, et al. M2det: a single-shot object detector based on multi-level feature pyramid network[C]∥Proceedings of the AAAI Conference on Artificial Intelligence, Honolulu, HI, USA, 2019: 9259-9266.
5	Cai Z, Vasconcelos N. Cascade R-CNN: high quality object detection and instance segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2019, 43(5): 1483-1498.
6	Shi S, Wang Z, Shi J, et al. From points to parts: 3D object detection from point cloud with part-aware and part-aggregation network[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020, 43(8): 2647-2664.
7	He C, Zeng H, Huang J, et al. Structure aware single-stage 3D object detection from point cloud[C]∥Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA, 2020: 11873-11882.
8	Zhou Y, Tuzel O. Voxelnet: end-to-end learning for point cloud based 3D object detection[C]∥Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, 2018: 4490-4499.
9	Yin T, Zhou X, Krahenbuhl P. Center-based 3D object detection and tracking[C]∥Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA,2021: 11784-11793.
10	Lang A H, Vora S, Caesar H, et al. Pointpillars: fast encoders for object detection from point clouds[C]∥Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA, 2019: 12697-12705.
11	Qi C R, Su H, Mo K, et al. Pointnet: deep learning on point sets for 3D classification and segmentation[C]∥Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA, 2017: 652-660.
12	Chen X, Ma H, Wan J, et al. Multi-view 3D object detection network for autonomous driving[C]∥Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA, 2017: 1907-1915.
13	Ku J, Mozifian M, Lee J, et al. Joint 3D proposal generation and object detection from view aggregation[C]∥2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Madrid, Spain, 2018: 1-8.
14	Liang M, Yang B, Wang S, et al. Deep continuous fusion for multi-sensor 3D object detection[C]∥Proceedings of the European Conference on Computer Vision, Munich, Germany, 2018: 641-656.
15	Liang M, Yang B, Chen Y, et al. Multi-task multi-sensor fusion for 3D object detection[C]∥Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA, 2019: 7345-7353.
16	Xu D, Anguelov D, Jain A. Pointfusion: deep sensor fusion for 3D bounding box estimation[C]∥Proceedings of the IEEE conference on computer vision and pattern recognition. Salt Lake City, UT, USA: IEEE, 2018: 244-253.
17	Xie L, Xiang C, Yu Z, et al. PI-RCNN: an efficient multi-sensor 3D object detector with point-based attentive cont-conv fusion module[C]∥Proceedings of the AAAI Conference on Artificial Intelligence, New York, NY, USA, 2020: 12460-12467.
18	Pang S, Morris D, Radha H. CLOCs: Camera- L i D A R object candidates fusion for 3D object detection[C]∥2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Las Vegas, NV, USA, 2020: 10386-10393.
19	Seif H G, Hu X. Autonomous driving in the iCity—HD maps as a key challenge of the automotive industry[J]. Engineering, 2016, 2(2): 159-162.
20	Chen Y F, Liu S Y, Liu M, et al. Motion planning with diffusion maps[C]∥2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Daejeon, Korea (South), 2016: 1423-1430.
21	Caesar H, Bankiti V, Lang A H, et al. Nuscenes: a multimodal dataset for autonomous driving[C]∥Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA, 2020: 11621-11631.
22	Gong L, Wang S, Zhang Y, et al. Lightweight map-enhanced 3D object detection and tracking for autonomous driving[C]∥12th Asia-Pacific Symposium on Internetware, Singapore Singapore, 2020: 165-174.
23	Yang B, Liang M, Urtasun R. HDNet: exploiting hd maps for 3D object detection[C]∥Conference on Robot Learning,Zürich, Switzerland, 2018: 146-155.
24	Yan Y, Mao Y, Li B. Second: sparsely embedded convolutional detection[J]. Sensors, 2018, 18(10): 3337-3354.
25	Liu Z, Mao H, Wu C Y, et al. A convnet for the 2020s[C]∥Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA, 2022: 11976-11986.
26	Liu Z, Lin Y, Cao Y, et al. Swin transformer: hierarchical vision transformer using shifted windows[C]∥Proceedings of the IEEE/CVF International Conference on Computer Vision, Montreal, QC, Canada, 2021: 10012-10022.
27	Ba J L, Kiros J R, Hinton G E. Layer normalization[J/OL]. [2022-09-20].
28	Ioffe S, Szegedy C. Batch normalization: accelerating deep network training by reducing internal covariate shift[C]∥International Conference on Machine Learning, Lile, France, 2015: 448-456.
29	Glorot X, Bordes A, Bengio Y. Deep sparse rectifier neural networks[J/OL].[2022-09-20]. .net/publication/215616967_Deep_Spars⁃e_Rectifier_Neural_Networks
30	Canny J. A computational approach to edge detection[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1986, 10(6): 679-698.
31	Zhang Z, Sabuncu M. Generalized cross entropy loss for training deep neural networks with noisy labels[J]. Advances in Neural Information Processing Systems, 2018, 31(5): 135-146.
32	Loshchilov I, Hutter F. Decoupled weight decay regularization[J/OL]. [2022-09-23].
33	He K, Zhang X, Ren S, et al. Deep residual learning for image recognition[C]∥Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA, 2016: 770-778.
34	杨怀江, 王二帅, 隋永新, 等. 简化型残差结构和快速深度残差网络[J]. 吉林大学学报: 工学版, 2022, 52(6): 1413-1421.
	Yang Huai-jiang, Wang Er-shuai, Sui Yong-xin, et al. Simplified residual structure and fast deep residual networks[J]. Journal of Jilin University (Engineering and Technology Edition), 2022, 52(6): 1413-1421.
35	申铉京, 张雪峰, 王玉, 等. 像素级卷积神经网络多聚焦图像融合算法[J]. 吉林大学学报: 工学版, 2022, 52(8): 1857-1864.
	Shen Xuan-jing, Zhang Xue-feng, Wang Yu, et al. Multi⁃focus image fusion algorithm based on pixel⁃level convolutional neural network[J]. Journal of Jilin University (Engineering and Technology Edition), 2022, 52(8): 1857-1864.

Related Articles 15

[1]	Ke HE,Hai-tao DING,Xuan-qi LAI,Nan XU,Kong-hui GUO. Wheel odometry error prediction model based on transformer [J]. Journal of Jilin University(Engineering and Technology Edition), 2023, 53(3): 653-662.
[2]	Zhen-yu WU,Xiao-fei LIU,Yi-pu WANG. Trajectory planning of unmanned system based on DKRRT^*⁃APF algorithm [J]. Journal of Jilin University(Engineering and Technology Edition), 2023, 53(3): 781-791.
[3]	Hong-yang PAN,Zhao LIU,Bo YANG,Geng SUN,Yan-heng LIU. Overview of swarm intelligence methods for unmanned aerial vehicle systems based on new⁃generation information technology [J]. Journal of Jilin University(Engineering and Technology Edition), 2023, 53(3): 629-642.
[4]	Peng GUO,Wen-chao ZHAO,Kun LEI. Dual⁃resource constrained flexible job shop optimal scheduling based on an improved Jaya algorithm [J]. Journal of Jilin University(Engineering and Technology Edition), 2023, 53(2): 480-487.
[5]	Jin-Zhen Liu,Guo-Hui Gao,Hui Xiong. Multi⁃scale attention network for brain tissue segmentation [J]. Journal of Jilin University(Engineering and Technology Edition), 2023, 53(2): 576-583.
[6]	Xiao-hu SHI,Jia-qi WU,Chun-guo WU,Shi CHENG,Xiao-hui WENG,Zhi-yong CHANG. Residual network based curve enhanced lane detection method [J]. Journal of Jilin University(Engineering and Technology Edition), 2023, 53(2): 584-592.
[7]	Xian-yu QI,Wei WANG,Lin WANG,Yu-fei ZHAO,Yan-peng DONG. Semantic topological map building with object semantic grid map [J]. Journal of Jilin University(Engineering and Technology Edition), 2023, 53(2): 569-575.
[8]	Feng-feng ZHOU,Hai-yang ZHU. SEE： sense EEG⁃based emotion algorithm via three⁃step feature selection strategy [J]. Journal of Jilin University(Engineering and Technology Edition), 2022, 52(8): 1834-1841.
[9]	Fu-heng QU,Tian-yu DING,Yang LU,Yong YANG,Ya-ting HU. Fast image codeword search algorithm based on neighborhood similarity [J]. Journal of Jilin University(Engineering and Technology Edition), 2022, 52(8): 1865-1871.
[10]	Tian BAI,Ming-wei XU,Si-ming LIU,Ji-an ZHANG,Zhe WANG. Dispute focus identification of pleading text based on deep neural network [J]. Journal of Jilin University(Engineering and Technology Edition), 2022, 52(8): 1872-1880.
[11]	Gui-he QIN,Jun-feng HUANG,Ming-hui SUN. Text input based on two⁃handed keyboard in virtual environment [J]. Journal of Jilin University(Engineering and Technology Edition), 2022, 52(8): 1881-1888.
[12]	Jun WANG,Yan-hui XU,Li LI. Data fusion privacy protection method with low energy consumption and integrity verification [J]. Journal of Jilin University(Engineering and Technology Edition), 2022, 52(7): 1657-1665.
[13]	Feng-feng ZHOU,Yi-chi ZHANG. Unsupervised feature engineering algorithm BioSAE based on sparse autoencoder [J]. Journal of Jilin University(Engineering and Technology Edition), 2022, 52(7): 1645-1656.
[14]	Yao-long KANG,Li-lu FENG,Jing-an ZHANG,Fu CHEN. Outlier mining algorithm for high dimensional categorical data streams based on spectral clustering [J]. Journal of Jilin University(Engineering and Technology Edition), 2022, 52(6): 1422-1427.
[15]	Wen-jun WANG,Yin-feng YU. Automatic completion algorithm for missing links in nowledge graph considering data sparsity [J]. Journal of Jilin University(Engineering and Technology Edition), 2022, 52(6): 1428-1433.

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed

Comments

Recommended 0

No Suggested Reading articles found!

方法	评价指标
方法	mAP/%	NDS/%
最大改进幅度%	1.74	1.17
单点云模型	41.17	56.84
融合基础模型	42.91	58.01

方法	评价指标
方法	mAP/%	NDS/%
最大改进幅度/%	0.60	0.53
融合基础模型	42.91	58.01
融合基础模型+ResNet18	43.02	58.18
融合基础模型+HFE	43.51	58.54

模型	评价指标
模型	总参数量/k	总浮点运算量/GFLOPs	总乘加操作量/GMAdd
最大改进幅度/%	-23	-13.43	-26.83
ResNet18	176	46.52	92.8
HFE模块	153	33.09	65.97

方法	nuScenes数据集详细类别精度										综合精度
方法	汽车	行人	公共汽车	障碍物	交通锥	卡车	拖车	摩托车	工程车	自行车	mAP/%	NDS/%
最大改进幅度/%	3.53	3.58	1.62	3.19	1.36	4.53	2.79	3.11	4.02	0.34	2.81	2.43
单点云模型	76.68	69.36	42.20	52.20	45.23	36.70	35.95	30.81	14.30	8.30	41.17	56.84
融合基础模型	78.22	71.50	42.08	55.04	45.50	39.52	37.33	33.77	17.19	8.98	42.91	58.01
融合基础模型+HFE	79.20	72.57	42.88	55.33	46.16	40.08	38.27	33.79	18.01	8.85	43.51	58.54
HME3D（本文）	80.21	72.94	43.82	55.39	46.59	41.23	38.74	33.92	18.32	8.64	43.98	59.27

3D object detection based on high⁃precision map enhancement

RICH HTML

PDF (PC)