基于多尺度候选融合与优化的三维目标检测算法

doi:10.13229/j.cnki.jdxbgxb.20230543

Abstract

Abstract:

To address the issues of target omission and the inclusion of a large number of background points in keypoint sampling for point cloud-based object detection， an improved algorithm based on the PV-RCNN network is introduced. This approach employs both a regional proposal fusion network and weighted non-maximum suppression （NMS） to merge proposals generated at various scales while eliminating redundancy. A segmentation network is utilized to segment foreground points from the original point cloud， and object center points are identified based on these proposals. Gaussian density functions are employed for regional density estimation， which assigns different sampling weights to solve the problem of difficult sampling in sparse areas. Experimental evaluations on the KITTI dataset indicate that the algorithm enhances the average precision at medium difficulty levels by 0.39%， 1.31%， and 0.63% for cars， pedestrians， and cyclists， respectively. Generalization experiments were also conducted on the Waymo open dataset. The results suggest that the introduced algorithm achieves higher accuracy compared to most of the existing 3D object detection networks.

Key words: computer version, 3D object detection, region proposal fusion, weighted non-maximum suppression, keypoint sampling

CLC Number:

TP391

Hua CAI,Yan-yang ZHENG,Qiang FU,Sheng-yu WANG,Wei-gang WANG,Zhi-yong MA. Three-dimensional object detection algorithm based on multi-scale candidate fusion and optimization[J].Journal of Jilin University(Engineering and Technology Edition), 2025, 55(2): 709-721.

Figures/Tables 15

Fig.1

Fig.2

Fig.3

Fig.4

Table 1

Table 2

Table 3

Table 4

Table 5

Table 6

Fig.5

Fig.6

Fig.7

Fig.8

Table 8

References 39

1	Qian R, Garg D, Wang Y, et al. End-to-end pseudo-lidar for image-based 3d object detection[C]∥Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, USA, 2020: 5881-5890.
2	Wang Z, Huang Z, Fu J, et al. Object as query: equipping any 2D object detector with 3D detection ability[J]. Arxiv Preprint, 2023, 1: No.230102364.
3	陶博, 颜伏伍, 尹智帅, 等. 基于高精度地图增强的三维目标检测算法[J]. 吉林大学学报: 工学版, 2023, 53(3): 802-809.
	Tao Bo, Yan Fu-wu, Yin Zhi-shuai, et al. 3D object detection algorithm based on high-precision map enhancement[J]. Journal of Jilin University (Engineering and Technology Edition), 2023, 53(3): 802-809.
4	Yang Z, Zhou Y, Chen Z, et al. 3D-man: 3D multi-frame attention network for object detection[C]∥Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Montreal, Canada, 2021: 1863-1872
5	Li Y, Yu A W, Meng T, et al. Deepfusion: lidar-camera deep fusion for multi-modal 3d object detection[C]∥Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, USA, 2022: 17182-17191.
6	才华, 寇婷婷, 杨依宁, 等. 基于轨迹优化的三维车辆多目标跟踪[J]. 吉林大学学报: 工学版, 2024, 54(8): 2338-2347.
	Cai Hua, Kou Ting-ting, Yang Yi-ning, et al. Three-dimensional vehicle multiple target tracking based on trajectory optimization[J]. Journal of Jilin University (Engineering and Technology Edition), 2024, 54(8): 2338-2347.
7	Zheng A, Zhang Y, Zhang X, et al. Progressive end-to-end object detection in crowded scenes[C]∥Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, USA, 2022: 857-866.
8	Redmon J, Divvala S, Girshick R, et al. You only look once: unified, real-time object detection[C]∥ 2016 IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, USA, 2016: 779-788.
9	Waleed A, Sherif A, Mahmoud Z, et al. Yolo3D: end-to-end real-time 3D oriented object bounding box detection from lidar point cloud[C]∥Computer Vision-ECCV 2018 Workshops, Munichi, Germany, 2018: 716-728.
10	Zhou Y, Sun P, Zhang Y, et al. End-to-end multi-view fusion for 3D object detection in lidar point clouds[C]∥Proceedings of the Conference on Robot Learning, Cambridge, USA, 2020: 923–932.
11	Zhou Y, Tuzel O. Voxelnet: end-to-end learning for point cloud based 3d object detection[C]∥Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, USA, 2018: 4490-4499.
12	Yan Y, Mao Y, Li B. Second: sparsely embedded convolutional detection[J]. Sensors, 2018, 18(10): No.18103337.
13	Qi C R, Su H, Mo K, et al. Pointnet: deep learning on point sets for 3d classification and segmentation[C]∥Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, USA, 2017: 652-660.
14	Qi C R, Yi L, Su H, et al. Pointnet++: deep hierarchical feature learning on point sets in a metric space[C]∥Proceedings of the 31st International Conference on Neural Information Processing Systems, Long Beach, USA, 2017: 5105-5114.
15	Qi C R, Liu W, Wu C, et al. Frustum pointnets for 3d object detection from rgb-d data[C]∥Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, USA, 2018: 918-927.
16	Yang Z, Sun Y, Liu S, et al. Std: sparse-to-dense 3d object detector for point cloud[C]∥Proceedings of The IEEE/CVF International Conference on Computer Vision, Seoul, South Korea, 2019: 1951-1960.
17	Shi S, Guo C, Jiang L, et al. PV-RCNN: point-voxel feature set abstraction for 3d object detection[C]∥Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, USA, 2020: 10529-10538.
18	Geiger A, Lenz P, Urtasun R. Are we ready for autonomous driving? the kitti vision benchmark suite[C]∥2012 IEEE Conference on Computer Vision and Pattern Recognition, Providence, Rhode Island,USA, 2012: 3354-3361.
19	Sun P, Kretzschmar H, Dotiwalla X, et al. Scalability in perception for autonomous driving: Waymo open dataset[C]∥Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, USA, 2020: 2446-2454.
20	Ye M, Xu S, Cao T. Hvnet: hybrid voxel network for lidar based 3D object detection[C]∥Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, USA, 2020: 1631-1640.
21	Shi S, Wang X, Li H. Pointrcnn: 3D object proposal generation and detection from point cloud[C]∥Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seoul, South Korea, 2019: 770-779.
22	Liu Z, Tang H, Lin Y, et al. Point-voxel CNN for efficient 3D deep learning[C]∥Proceedings of the 33rd International Conference on Neural Information Processing Systems, Vancouver, Canada, 2019: 965-975.
23	田枫, 姜文文, 刘芳, 等. 混合体素与原始点云的三维目标检测方法[J]. 重庆理工大学学报: 自然科学, 2022, 36(11): 108-117.
	Tian Feng, Jiang Wen-wen, Liu Fang, et al. Hybrid element and original point cloud 3D target detection method [J]. Journal of Chongqing University of Technology (Natural Science), 2022, 36(11):108-117.
24	Shi S, Jiang L, Deng J, et al. PV-RCNN++: point-voxel feature set abstraction with local vector representation for 3D object detection[J]. International Journal of Computer Vision, 2023, 131(2): 531-551.
25	车运龙, 袁亮, 孙丽慧. 基于强语义关键点采样的三维目标检测方法[J]. 计算机工程与应用, 2024, 60(9): 254-260.
	Che Yun-long, Yuan Liang, Sun Li-hui, et al. 3D object detection method based on strong semantic key point sampling[J]. Computer Engineering and Applications, 2024, 60(9): 254-260.
26	He C, Zeng H, Huang J, et al. Structure aware single-stage 3D object detection from point cloud[C]∥Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, USA, 2020: 11873-11882.
27	Zhang Y, Hu Q, Xu G, et al. Not all points are equal: learning highly efficient point-based detectors for 3D lidar point clouds[C]∥Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, USA, 2022: 18953-18962.
28	Lin T Y, Goyal P, Girshick R, et al. Focal loss for dense object detection[C]∥Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy, 2017: 2980-2988.
29	Lang A H, Vora S, Caesar H, et al. Pointpillars: fast encoders for object detection from point clouds [C]∥Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seoul, South Korea, 2019: 12697-12705.
30	Shi W, Rajkumar R. Point-GNN: graph neural network for 3d object detection in a point cloud[C]∥Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, USA, 2020: 1711-1719.
31	Liu Z, Zhao X, Huang T, et al. Tanet: robust 3D object detection from point clouds with triple attention[J]. Proceedings of the AAAI Conference on Artificial Intelligence, 2020, 34(7): 11677-11684.
32	Shi S, Wang Z, Shi J, et al. From points to parts: 3D object detection from point cloud with part-aware and part-aggregation network[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020, 43(8): 2647-2664.
33	Deng J, Shi S, Li P, et al. Voxel R-Cnn: towards high performance voxel-based 3d object detection[J]. Proceedings of the AAAI Conference on Artificial Intelligence, 2021, 35(2): 1201-1209.
34	Song N, Jiang T, Yao J. JPV-Net: joint point-voxel representations for accurate 3D object detection[J]. Proceedings of the AAAI Conference on Artificial Intelligence, 2022, 36(2): 2271-2279.
35	Yang Z, Jiang L, Sun Y, et al. A unified query-based paradigm for point cloud understanding[C]∥Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, USA, 2022: 8541-8551.
36	Yin T, Zhou X, Krahenbuhl P. Center-based 3D object detection and tracking[C]∥Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Nashville, USA, 2021: 11784-11793.
37	Zhou Z, Zhao X, Wang Y, et al. Centerformer: center-based transformer for 3D object detection[C]∥European Conference on Computer Vision, Tel Aviv, Israel, 2022: 496-513.
38	He C, Li R, Li S, et al. Voxel set transformer: a set-to-set approach to 3D object detection from point clouds[C]∥2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, USA, 2022: 8417-8427.
39	Sheng H, Cai S, Liu Y, et al. Improving 3D object detection with channel-wise transformer[C]∥Proceedings of the IEEE/CVF International Conference on Computer Vision, Montreal, Canada, 2021: 2743-2752.

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed

Comments

Recommended 0

No Suggested Reading articles found!

等级	简单	中等	困难
最小边界框高度	40像素	25像素	25像素
最大遮挡等级	完全可见	部分遮挡	难以看清
最大截断率	15%	30%	50%

算法	类型	3D 车辆（IoU=0.7）/%			3D行人（IoU=0.5）/%			3D 骑行者（IoU=0.5）/%
算法	类型	简单	中等	困难	简单	中等	困难	简单	中等	困难
VoxelNet^［11］	一阶段	77.47	65.11	57.73	39.48	33.69	31.50	61.22	48.36	44.37
SECOND^［12］	一阶段	84.65	75.96	68.71	45.31	35.52	33.14	75.83	60.82	53.67
PointPillars^［29］	一阶段	82.58	74.31	68.99	51.45	41.92	38.89	77.10	58.65	51.92
Point-GNN^［30］	一阶段	88.33	79.47	72.29	51.92	43.77	40.14	78.60	63.48	57.08
IAvSSD^［27］	一阶段	88.34	80.13	75.04	46.51	39.03	35.61	78.35	61.94	55.70
TANet^［31］	二阶段	84.39	75.94	68.82	53.72	44.34	40.49	75.70	59.44	52.53
Part-A^2［32］	二阶段	87.81	78.49	73.51	53.10	43.35	40.06	79.17	63.52	56.93
PointRCNN^［21］	二阶段	86.96	75.64	70.70	47.98	39.37	36.01	74.96	58.82	52.53
STD^［16］	二阶段	87.95	79.71	75.09	53.29	42.47	38.35	78.69	61.59	55.30
PV-RCNN^［17］	二阶段	90.25	81.43	76.82	52.17	43.29	40.29	78.60	63.71	57.65
本文	二阶段	90.32	81.82	77.11	53.86	44.60	40.84	79.52	64.34	58.03

算法	年份	mAP_3D/%
算法	年份	简单	中等	困难
SECOND^［12］	2018	88.61	78.62	77.22
PointPillars^［29］	2019	86.62	76.06	68.91
STD^［16］	2019	89.70	79.80	79.30
PointRCNN^［21］	2019	88.88	78.63	77.38
Part-A^{2 ［30］}	2020	89.47	79.47	78.54
PV-RCNN^［17］	2020	89.35	83.69	78.70
VoxelRCNN^［33］	2021	89.41	84.52	78.93
JPV-Net^［34］	2022	89.71	84.61	79.09
本文	2025	90.54	84.72	79.81

算法	车辆（mAP/mAPH）/%		行人（mAP/mAPH）/%		骑行者（mAP/mAPH）/%
算法	LEVEL 1	LEVEL 2	LEVEL 1	LEVEL 2	LEVEL 1	LEVEL 2
SECOND^［12］	72.3/71.7	63.9/63.3	68.7/58.2	60.7/51.3	60.6/59.3	58.3/57.0
PointPillars^［29］	72.1/71.5	63.6/63.1	70.6/56.7	62.8/50.3	64.4/62.3	61.9/59.9
Centerpoint^［36］	76.6/76.0	68.9/68.4	79.0/73.4	71.0/65.8	72.1/71.0	69.5/68.5
IA-SSD^［27］	70.5/69.7	61.6/60.9	69.4/58.5	60.3/50.7	67.7/65.3	65.0/62.7
CenterFormer^［37］	75.2/74.7	70.2/69.7	78.6/73.0	73.6/68.3	72.3/71.3	69.8/68.8
VoxSeT^［38］	74.5/74.0	66.0/65.6	80.0/72.4	72.5/65.4	71.6/70.3	69.0/67.8
PV-RCNN^［17］	77.5/76.9	69.0/68.4	75.0/65.6	66.0/57.6	67.8/66.4	65.4/64.0
本文	78.5/78.1	72.5/71.8	80.1/75.3	73.8/67.8	74.3/73.6	70.9/68.7

模型	年份	车辆 3D mAP （IoU=0.7）
模型	年份	0~30 m	30~50 m	50 m-Inf
PV-RCNN^［17］	2020	91.92	69.21	42.17
Voxel-RCNN^［33］	2021	92.49	74.09	53.15
CT3D^［39］	2021	92.51	75.07	55.36
VoxSeT^［38］	2022	91.13	75.75	54.23
本文	2025	92.10	75.81	55.42

Three-dimensional object detection algorithm based on multi-scale candidate fusion and optimization

RICH HTML

PDF (PC)

Abstract

Cite this article

share this article

Figures/Tables 15

References 39

Related Articles 3

Metrics

Comments

Recommended 0

算法	区域候选融合	加权NMS	区域密度采样	mAP（中等）/%
算法	区域候选融合	加权NMS	区域密度采样	车辆	行人	骑行者
基础网络（PV-RCNN）				81.43	43.29	63.71
实验网络1	√			81.60	44.24	64.05
实验网络2	√	√		81.76	44.35	64.09
本文算法	√	√	√	81.82	44.70	64.34

[1]	Hua CAI,Ting-ting KOU,Yi-ning YANG,Zhi-yong MA,Wei-gang WANG,Jun-xi SUN. Three-dimensional vehicle multi-target tracking based on trajectory optimization [J]. Journal of Jilin University(Engineering and Technology Edition), 2024, 54(8): 2338-2347.
[2]	Xin CHENG,Sheng-xian LIU,Jing-mei ZHOU,Zhou ZHOU,Xiang-mo ZHAO. 3D object detection algorithm fusing dense connectivity and Gaussian distance [J]. Journal of Jilin University(Engineering and Technology Edition), 2024, 54(12): 3589-3600.
[3]	Bo TAO,Fu-wu YAN,Zhi-shuai YIN,Dong-mei WU. 3D object detection based on high⁃precision map enhancement [J]. Journal of Jilin University(Engineering and Technology Edition), 2023, 53(3): 802-809.