基于多尺度候选融合与优化的三维目标检测算法

doi:10.13229/j.cnki.jdxbgxb.20230543

摘要/Abstract

摘要：

为了改善点云场景下的检测任务中，基于单一低分辨特征图生成的候选框容易造成目标丢失和关键点采样过程中引入大量背景点的问题，本文提出了一种基于PV-RCNN网络的改进算法。通过区域候选融合网络和加权非极大值抑制融合不同尺度下的候选框并消除冗余。利用分割网络对原始点云进行前景点分割，并根据候选框确定目标中心点位置，利用高斯密度函数进行区域密度估计得到不同的采样权重以解决稀疏区域采样困难的问题。本文使用KITTI数据集进行实验评估，在汽车、行人和骑行者中等难度下的平均精度分别较基线算法提升0.39%、1.31%和0.63%，并同样在Waymo open数据集上进行泛化实验。实验结果证明本文算法与目前大部分三维目标检测算法相比取得更高的检测精度。

关键词: 计算机视觉, 三维目标检测, 区域候选融合, 加权非极大值抑制, 关键点采样

Abstract:

To address the issues of target omission and the inclusion of a large number of background points in keypoint sampling for point cloud-based object detection， an improved algorithm based on the PV-RCNN network is introduced. This approach employs both a regional proposal fusion network and weighted non-maximum suppression （NMS） to merge proposals generated at various scales while eliminating redundancy. A segmentation network is utilized to segment foreground points from the original point cloud， and object center points are identified based on these proposals. Gaussian density functions are employed for regional density estimation， which assigns different sampling weights to solve the problem of difficult sampling in sparse areas. Experimental evaluations on the KITTI dataset indicate that the algorithm enhances the average precision at medium difficulty levels by 0.39%， 1.31%， and 0.63% for cars， pedestrians， and cyclists， respectively. Generalization experiments were also conducted on the Waymo open dataset. The results suggest that the introduced algorithm achieves higher accuracy compared to most of the existing 3D object detection networks.

Key words: computer version, 3D object detection, region proposal fusion, weighted non-maximum suppression, keypoint sampling

中图分类号:

TP391

才华,郑延阳,付强,王晟宇,王伟刚,马智勇. 基于多尺度候选融合与优化的三维目标检测算法[J]. 吉林大学学报(工学版), 2025, 55(2): 709-721.

Hua CAI,Yan-yang ZHENG,Qiang FU,Sheng-yu WANG,Wei-gang WANG,Zhi-yong MA. Three-dimensional object detection algorithm based on multi-scale candidate fusion and optimization[J]. Journal of Jilin University(Engineering and Technology Edition), 2025, 55(2): 709-721.

图/表 15

图1

图2

图3

图4

表1

表2

表3

表4

表5

表6

图5

图6

图7

图8

表8

参考文献 39

1	Qian R, Garg D, Wang Y, et al. End-to-end pseudo-lidar for image-based 3d object detection[C]∥Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, USA, 2020: 5881-5890.
2	Wang Z, Huang Z, Fu J, et al. Object as query: equipping any 2D object detector with 3D detection ability[J]. Arxiv Preprint, 2023, 1: No.230102364.
3	陶博, 颜伏伍, 尹智帅, 等. 基于高精度地图增强的三维目标检测算法[J]. 吉林大学学报: 工学版, 2023, 53(3): 802-809.
	Tao Bo, Yan Fu-wu, Yin Zhi-shuai, et al. 3D object detection algorithm based on high-precision map enhancement[J]. Journal of Jilin University (Engineering and Technology Edition), 2023, 53(3): 802-809.
4	Yang Z, Zhou Y, Chen Z, et al. 3D-man: 3D multi-frame attention network for object detection[C]∥Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Montreal, Canada, 2021: 1863-1872
5	Li Y, Yu A W, Meng T, et al. Deepfusion: lidar-camera deep fusion for multi-modal 3d object detection[C]∥Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, USA, 2022: 17182-17191.
6	才华, 寇婷婷, 杨依宁, 等. 基于轨迹优化的三维车辆多目标跟踪[J]. 吉林大学学报: 工学版, 2024, 54(8): 2338-2347.
	Cai Hua, Kou Ting-ting, Yang Yi-ning, et al. Three-dimensional vehicle multiple target tracking based on trajectory optimization[J]. Journal of Jilin University (Engineering and Technology Edition), 2024, 54(8): 2338-2347.
7	Zheng A, Zhang Y, Zhang X, et al. Progressive end-to-end object detection in crowded scenes[C]∥Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, USA, 2022: 857-866.
8	Redmon J, Divvala S, Girshick R, et al. You only look once: unified, real-time object detection[C]∥ 2016 IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, USA, 2016: 779-788.
9	Waleed A, Sherif A, Mahmoud Z, et al. Yolo3D: end-to-end real-time 3D oriented object bounding box detection from lidar point cloud[C]∥Computer Vision-ECCV 2018 Workshops, Munichi, Germany, 2018: 716-728.
10	Zhou Y, Sun P, Zhang Y, et al. End-to-end multi-view fusion for 3D object detection in lidar point clouds[C]∥Proceedings of the Conference on Robot Learning, Cambridge, USA, 2020: 923–932.
11	Zhou Y, Tuzel O. Voxelnet: end-to-end learning for point cloud based 3d object detection[C]∥Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, USA, 2018: 4490-4499.
12	Yan Y, Mao Y, Li B. Second: sparsely embedded convolutional detection[J]. Sensors, 2018, 18(10): No.18103337.
13	Qi C R, Su H, Mo K, et al. Pointnet: deep learning on point sets for 3d classification and segmentation[C]∥Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, USA, 2017: 652-660.
14	Qi C R, Yi L, Su H, et al. Pointnet++: deep hierarchical feature learning on point sets in a metric space[C]∥Proceedings of the 31st International Conference on Neural Information Processing Systems, Long Beach, USA, 2017: 5105-5114.
15	Qi C R, Liu W, Wu C, et al. Frustum pointnets for 3d object detection from rgb-d data[C]∥Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, USA, 2018: 918-927.
16	Yang Z, Sun Y, Liu S, et al. Std: sparse-to-dense 3d object detector for point cloud[C]∥Proceedings of The IEEE/CVF International Conference on Computer Vision, Seoul, South Korea, 2019: 1951-1960.
17	Shi S, Guo C, Jiang L, et al. PV-RCNN: point-voxel feature set abstraction for 3d object detection[C]∥Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, USA, 2020: 10529-10538.
18	Geiger A, Lenz P, Urtasun R. Are we ready for autonomous driving? the kitti vision benchmark suite[C]∥2012 IEEE Conference on Computer Vision and Pattern Recognition, Providence, Rhode Island,USA, 2012: 3354-3361.
19	Sun P, Kretzschmar H, Dotiwalla X, et al. Scalability in perception for autonomous driving: Waymo open dataset[C]∥Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, USA, 2020: 2446-2454.
20	Ye M, Xu S, Cao T. Hvnet: hybrid voxel network for lidar based 3D object detection[C]∥Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, USA, 2020: 1631-1640.
21	Shi S, Wang X, Li H. Pointrcnn: 3D object proposal generation and detection from point cloud[C]∥Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seoul, South Korea, 2019: 770-779.
22	Liu Z, Tang H, Lin Y, et al. Point-voxel CNN for efficient 3D deep learning[C]∥Proceedings of the 33rd International Conference on Neural Information Processing Systems, Vancouver, Canada, 2019: 965-975.
23	田枫, 姜文文, 刘芳, 等. 混合体素与原始点云的三维目标检测方法[J]. 重庆理工大学学报: 自然科学, 2022, 36(11): 108-117.
	Tian Feng, Jiang Wen-wen, Liu Fang, et al. Hybrid element and original point cloud 3D target detection method [J]. Journal of Chongqing University of Technology (Natural Science), 2022, 36(11):108-117.
24	Shi S, Jiang L, Deng J, et al. PV-RCNN++: point-voxel feature set abstraction with local vector representation for 3D object detection[J]. International Journal of Computer Vision, 2023, 131(2): 531-551.
25	车运龙, 袁亮, 孙丽慧. 基于强语义关键点采样的三维目标检测方法[J]. 计算机工程与应用, 2024, 60(9): 254-260.
	Che Yun-long, Yuan Liang, Sun Li-hui, et al. 3D object detection method based on strong semantic key point sampling[J]. Computer Engineering and Applications, 2024, 60(9): 254-260.
26	He C, Zeng H, Huang J, et al. Structure aware single-stage 3D object detection from point cloud[C]∥Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, USA, 2020: 11873-11882.
27	Zhang Y, Hu Q, Xu G, et al. Not all points are equal: learning highly efficient point-based detectors for 3D lidar point clouds[C]∥Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, USA, 2022: 18953-18962.
28	Lin T Y, Goyal P, Girshick R, et al. Focal loss for dense object detection[C]∥Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy, 2017: 2980-2988.
29	Lang A H, Vora S, Caesar H, et al. Pointpillars: fast encoders for object detection from point clouds [C]∥Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seoul, South Korea, 2019: 12697-12705.
30	Shi W, Rajkumar R. Point-GNN: graph neural network for 3d object detection in a point cloud[C]∥Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, USA, 2020: 1711-1719.
31	Liu Z, Zhao X, Huang T, et al. Tanet: robust 3D object detection from point clouds with triple attention[J]. Proceedings of the AAAI Conference on Artificial Intelligence, 2020, 34(7): 11677-11684.
32	Shi S, Wang Z, Shi J, et al. From points to parts: 3D object detection from point cloud with part-aware and part-aggregation network[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020, 43(8): 2647-2664.
33	Deng J, Shi S, Li P, et al. Voxel R-Cnn: towards high performance voxel-based 3d object detection[J]. Proceedings of the AAAI Conference on Artificial Intelligence, 2021, 35(2): 1201-1209.
34	Song N, Jiang T, Yao J. JPV-Net: joint point-voxel representations for accurate 3D object detection[J]. Proceedings of the AAAI Conference on Artificial Intelligence, 2022, 36(2): 2271-2279.
35	Yang Z, Jiang L, Sun Y, et al. A unified query-based paradigm for point cloud understanding[C]∥Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, USA, 2022: 8541-8551.
36	Yin T, Zhou X, Krahenbuhl P. Center-based 3D object detection and tracking[C]∥Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Nashville, USA, 2021: 11784-11793.
37	Zhou Z, Zhao X, Wang Y, et al. Centerformer: center-based transformer for 3D object detection[C]∥European Conference on Computer Vision, Tel Aviv, Israel, 2022: 496-513.
38	He C, Li R, Li S, et al. Voxel set transformer: a set-to-set approach to 3D object detection from point clouds[C]∥2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, USA, 2022: 8417-8427.
39	Sheng H, Cai S, Liu Y, et al. Improving 3D object detection with channel-wise transformer[C]∥Proceedings of the IEEE/CVF International Conference on Computer Vision, Montreal, Canada, 2021: 2743-2752.

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed

等级	简单	中等	困难
最小边界框高度	40像素	25像素	25像素
最大遮挡等级	完全可见	部分遮挡	难以看清
最大截断率	15%	30%	50%

算法	类型	3D 车辆（IoU=0.7）/%			3D行人（IoU=0.5）/%			3D 骑行者（IoU=0.5）/%
算法	类型	简单	中等	困难	简单	中等	困难	简单	中等	困难
VoxelNet^［11］	一阶段	77.47	65.11	57.73	39.48	33.69	31.50	61.22	48.36	44.37
SECOND^［12］	一阶段	84.65	75.96	68.71	45.31	35.52	33.14	75.83	60.82	53.67
PointPillars^［29］	一阶段	82.58	74.31	68.99	51.45	41.92	38.89	77.10	58.65	51.92
Point-GNN^［30］	一阶段	88.33	79.47	72.29	51.92	43.77	40.14	78.60	63.48	57.08
IAvSSD^［27］	一阶段	88.34	80.13	75.04	46.51	39.03	35.61	78.35	61.94	55.70
TANet^［31］	二阶段	84.39	75.94	68.82	53.72	44.34	40.49	75.70	59.44	52.53
Part-A^2［32］	二阶段	87.81	78.49	73.51	53.10	43.35	40.06	79.17	63.52	56.93
PointRCNN^［21］	二阶段	86.96	75.64	70.70	47.98	39.37	36.01	74.96	58.82	52.53
STD^［16］	二阶段	87.95	79.71	75.09	53.29	42.47	38.35	78.69	61.59	55.30
PV-RCNN^［17］	二阶段	90.25	81.43	76.82	52.17	43.29	40.29	78.60	63.71	57.65
本文	二阶段	90.32	81.82	77.11	53.86	44.60	40.84	79.52	64.34	58.03

算法	年份	mAP_3D/%
算法	年份	简单	中等	困难
SECOND^［12］	2018	88.61	78.62	77.22
PointPillars^［29］	2019	86.62	76.06	68.91
STD^［16］	2019	89.70	79.80	79.30
PointRCNN^［21］	2019	88.88	78.63	77.38
Part-A^{2 ［30］}	2020	89.47	79.47	78.54
PV-RCNN^［17］	2020	89.35	83.69	78.70
VoxelRCNN^［33］	2021	89.41	84.52	78.93
JPV-Net^［34］	2022	89.71	84.61	79.09
本文	2025	90.54	84.72	79.81

算法	车辆（mAP/mAPH）/%		行人（mAP/mAPH）/%		骑行者（mAP/mAPH）/%
算法	LEVEL 1	LEVEL 2	LEVEL 1	LEVEL 2	LEVEL 1	LEVEL 2
SECOND^［12］	72.3/71.7	63.9/63.3	68.7/58.2	60.7/51.3	60.6/59.3	58.3/57.0
PointPillars^［29］	72.1/71.5	63.6/63.1	70.6/56.7	62.8/50.3	64.4/62.3	61.9/59.9
Centerpoint^［36］	76.6/76.0	68.9/68.4	79.0/73.4	71.0/65.8	72.1/71.0	69.5/68.5
IA-SSD^［27］	70.5/69.7	61.6/60.9	69.4/58.5	60.3/50.7	67.7/65.3	65.0/62.7
CenterFormer^［37］	75.2/74.7	70.2/69.7	78.6/73.0	73.6/68.3	72.3/71.3	69.8/68.8
VoxSeT^［38］	74.5/74.0	66.0/65.6	80.0/72.4	72.5/65.4	71.6/70.3	69.0/67.8
PV-RCNN^［17］	77.5/76.9	69.0/68.4	75.0/65.6	66.0/57.6	67.8/66.4	65.4/64.0
本文	78.5/78.1	72.5/71.8	80.1/75.3	73.8/67.8	74.3/73.6	70.9/68.7

模型	年份	车辆 3D mAP （IoU=0.7）
模型	年份	0~30 m	30~50 m	50 m-Inf
PV-RCNN^［17］	2020	91.92	69.21	42.17
Voxel-RCNN^［33］	2021	92.49	74.09	53.15
CT3D^［39］	2021	92.51	75.07	55.36
VoxSeT^［38］	2022	91.13	75.75	54.23
本文	2025	92.10	75.81	55.42