吉林大学学报(工学版) ›› 2021, Vol. 51 ›› Issue (4): 1437-1446.doi: 10.13229/j.cnki.jdxbgxb20200380

• 计算机科学与技术 • 上一篇    

面向动态场景复合深度学习与并行计算的DG-SLAM算法

兰凤崇1,2(),李继文1,2,陈吉清1,2()   

  1. 1.华南理工大学 机械与汽车工程学院,广州 510640
    2.华南理工大学 广东省汽车工程重点实验室,广州 510640
  • 收稿日期:2020-05-30 出版日期:2021-07-01 发布日期:2021-07-14
  • 通讯作者: 陈吉清 E-mail:fclan@scut.edu.cn;chjq@scut.edu.cn
  • 作者简介:兰凤崇(1959-),男,教授,博士.研究方向:车身结构与安全技术. E-mail:fclan@scut.edu.cn
  • 基金资助:
    国家自然科学基金项目(51775193);广东省科技计划项目(2015B010137002)

DG-SLAM algorithm for dynamic scene compound deep learning and parallel computing

Feng-chong LAN1,2(),Ji-wen LI1,2,Ji-qing CHEN1,2()   

  1. 1.School of Mechanical & Automotive Engineering,South China University of Technology,Guangzhou 510640,China
    2.Guangdong Key Laboratory of Automotive Engineering,South China University of Technology,Guangzhou 510640,China
  • Received:2020-05-30 Online:2021-07-01 Published:2021-07-14
  • Contact: Ji-qing CHEN E-mail:fclan@scut.edu.cn;chjq@scut.edu.cn

摘要:

针对现有的同时定位与建图(SLAM)算法实时性不高和在动态环境中定位精度会大幅降低的缺点,提出了一种复合深度学习与并行计算的DG-SLAM算法。采用基于深度学习的目标检测算法检测出行驶环境中的动态物体,在ORB-SLAM2图像帧间匹配前剔除动态物体特征点,降低动态物体对SLAM定位精度的影响;在ORB-SLAM2跟踪局部地图中采用三维空间下内部点的判别方法区分内点和外点,建立GPU并行计算模型以高效搜索局部地图点;利用Saturated核函数作用于重投影误差项的二范数平方和,确保局部地图优化位姿时重投影误差的并行计算。在KITII数据集上进行了算法验证,结果表明,DG-SLAM具有较高跟踪精度,且平均计算效率相同情况下对比ORB-SLAM2高3.4倍以上,超过85帧/s,可实现自动驾驶车辆在动态环境下SLAM系统的稳定运行。

关键词: 车辆工程, 同时定位与建图, 深度学习, 目标检测, 并行计算

Abstract:

In view of the disadvantages of the existing simultaneous localization and mapping (SLAM) algorithm, which has low real-time performance and the positioning accuracy is greatly reduced in dynamic environment, a DG-SLAM algorithm based on deep learning and GPU parallel computing was proposed. The deep learning-based object detection algorithm was introduced to detect dynamic objects in the driving environment, and the feature points of dynamic objects were removed before the matching of image frames, so as to eliminate the impact of mismatching dynamic objects on the positioning accuracy of SLAM system. In the tracking of local maps of ORB-SLAM2, the discriminant method of 3D interior points was used to distinguish the inner points and outer points, and the GPU parallel computing model was established to efficiently search the local map points. The Saturated kernel function was used to minimize the reprojection error of the two norm terms to ensure the parallel calculation of the reprojection error when the local map was optimizing. The algorithm was verified on the KITII dataset, DG-SLAM has high tracking accuracy, and the average calculation efficiency was more than 3.4 times faster than that of ORB-SLAM2 system under the same conditions, more than 85 frames per second, which could realize efficient and high precision SLAM in dynamic scene.

Key words: vehicle engineering, simultaneous localization and mapping, deep learning, object detection, parallel computing

中图分类号: 

  • TP391.4

图1

DG-SLAM总体系统构架"

图2

ORB-SLAM2定位流程"

图3

DG-SLAM定位部分并行计算流程"

图4

不同残差阈值下饱和核函数变化趋势"

图5

SSD网络结构"

图6

剔除动态特征点前、后的数据关联对比图"

图7

KITTI数据集道路类型"

图8

DG-SLAM与ORB-SLAM2在KITTI 01数据集上位置估计对比"

图9

DG-SLAM与ORB-SLAM2在KITTI 01数据集上姿态角估计对比"

图10

ORB-SLAM在KITTI 01数据集上测试的轨迹误差图"

图11

DG-SLAM在KITTI 01数据集上测试的轨迹误差图"

图12

DG-SLAM和ORB-SLAM2绝对位姿误差"

表1

KITTI数据集上的测试精度"

序列编号场景类型轨迹长度/mRMSE
ORB?SLAM2DG?SLAM
平均值--20.73717.692
00城市支路714.269.6169.560
01高速公路2453.2011.7461.666
02城市主干路5067.2347.15028.354
03城市主干路560.880.9200.918
04城市快速路393.640.7030.423
05城市支路2205.5744.32444.532
06城市支路1232.8729.34927.258
07城市支路694.6917.87316.828
08城市支路3222.7955.18452.689
09城市主干路1705.0558.73349.568
10城市支路919.516.3846.356

图13

DG-SLAM和ORB-SLAM2在KIITI 04上APE对比"

图14

DG-SLAM和ORB-SLAM2在KIITI 02上APE对比"

图15

DG-SLAM和ORB-SLAM2在KIITI 03上APE对比"

图16

DG-SLAM与ORB-SLAM2在KITTI 01数据集上耗时比较"

1 Yousif K, Bab-Hadiashar A, Hoseinnezhad R. An overview to visual odometry and visual SLAM: applications to mobile robotics[J]. Intelligent Industrial Systems, 2015, 1(4): 289-311.
2 邸凯昌,万文辉,赵红颖,等.视觉SLAM技术的进展与应用[J].测绘学报,2018,47(6):770-779.
Di Kai-chang, Wan Wen-hui, Zhao Hong-ying, et al. Progress and applications of visual SLAM[J]. Acta Geodaetica et Cartographica Sinica, 2018,47(6):770-779.
3 高成强,张云洲,王晓哲, 等. 面向室内动态环境的半直接法RGB-D SLAM算法[J].机器人, 2019, 41(3): 372-383.
Gao Cheng-qiang, Zhang Yun-zhou, Wang Xiao-Zhe, et al. Semi-direct RGB-D SLAM algorithm for dynamic indoor environments[J]. Robot, 2019, 41(3): 372-383.
4 Newcombe R A, Fox D, Seitz S M. Dynamic fusion: reconstruction and tracking of non-rigid scenes in real-time[C]∥The IEEE Conference on Computer Vision and Pattern Recognition(CVPR), Portland, Oregon, 2015: 343-352
5 Saputra M R U, Markham A, Trigoni N. Visual SLAM andstructure from motion in dynamic environments[J]. ACM Computing Surveys, 2018, 51(2): 1-36.
6 Klein G, Murray D. Parallel tracking and mapping for small AR workspaces[C]∥The 6th IEEE and ACM International Symposium on Mixed and Augmented Reality, Arlington, USA, 2007: 225-234.
7 Zhou Yong-long, Ma Kui-zhi, Jiang Xiang, et al. Parallelization and optimization of sift on gpu using cuda[C]∥The IEEE 10th International Conference on High Performance Computing and Communications & IEEE International Conference on Embedded and Ubiquitous Computing, Dalian, China, 2008: 1351-1358.
8 Bescos B, Fácil J M, Civera J, et al. DynaSLAM: Tracking, mapping, and inpainting in dynamic scenes[J]. IEEE Robotics and Automation Letters, 2018, 3(4): 4076-4083.
9 Kawewong A, Tongprasit N, Tangruamsub S, et al. Online and incremental appearance-based SLAM in highly dynamic environments[J]. International Journal of Robotics Research, 2011, 30(1):33-55.
10 Zhong F, Wang S, Zhang Z, et al. Detect-slam: Making object detection and slam mutually beneficial[C]∥IEEE Winter Conference on Applications of Computer Vision (WACV), Lake Tahoe, USA, 2018: 1001-1010.
11 Hosseinyalamdary S. Deep Kalman filter: simultaneous multi-sensor integration and modelling; a GNSS/IMU case study[J]. Sensors, 2018, 18(5): 1316.
12 Mur-Artal R, Tardós J D. ORB-SLAM2: an open-source slam system for monocular, stereo, and RGB-D cameras[J]. IEEE Transactions on Robotics, 2017, 33(5): 1255-1262.
13 Mur-Artal R, Montiel J M M, Tardos J D. ORB-SLAM: a versatile and accurate monocular SLAM system[J]. IEEE Transactions on Robotics, 2015, 31(5): 1147-1163.
14 李涛, 董前琨, 张帅, 等. 基于线程池的GPU任务并行计算模式研究[J]. 计算机学报, 2018, 41(10): 2175-2192.
Li Tao, Dong Qian-kun, Zang Shuai, et al. GPU task parallel computing paradigm based on thread pool model[J]. Chinese Journal of Computers, 2018, 41(10): 2175-2192.
15 Rublee E, Rabaud V, Konolige K, et al. ORB: an efficient alternative to SIFT or SURF[C]∥The International Conference on Computer Vision, Barcelona, Spain, 2011: 2564-2571.
16 Bay H, Tuytelaars T, van Gool L. Surf: speeded up robust features[C]∥European Conference on Computer Vision, Berlin, Germany, 2006: 404-417.
17 Lowe D G. Distinctive image features from scale-invariant keypoints[J]. International Journal of Computer Vision, 2004, 60(2): 91-110.
18 Lepetit V, Moreno-Noguer F, Fua P. EPnP: an accurate on solution to the PnP problem[J]. International Journal of Computer Vision, 2009, 81(2): 155-166.
19 Liu W, Anguelov D, Erhan D, et al. SSD: single shot multibox detector[C]∥European Conference on Computer Vision, Amsterdam, The Netherlands, 2016: 21-37.
20 Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition [J]. arXiv preprint arXiv:, 2014.
21 Geiger A, Lenz P, Stiller C, et al. Vision meets robotics: the KITTI dataset[J]. The International Journal of Robotics Research, 2013, 32(11): 1231-1237.
[1] 陈鑫,于贵申,张彪,潘凯旋,杨立飞. 搅拌摩擦点焊接头拉伸-剪切行为的等效建模[J]. 吉林大学学报(工学版), 2021, 51(4): 1190-1197.
[2] 潘德伦,冀隽,张跃进. 基于运动矢量空间编码的视频监控动态目标检测方法[J]. 吉林大学学报(工学版), 2021, 51(4): 1370-1374.
[3] 杨建,夏琦,周海超,王国林. 修正胎体弦轮廓载重子午线轮胎的降噪机理[J]. 吉林大学学报(工学版), 2021, 51(4): 1198-1203.
[4] 龙江启,向锦涛,俞平,王骏骋. 适用于非线性主动悬架滑模控制的线性干扰观测器[J]. 吉林大学学报(工学版), 2021, 51(4): 1230-1240.
[5] 陈雪云,许韬,黄小巧. 基于条件生成对抗网络的医学细胞图像生成检测方法[J]. 吉林大学学报(工学版), 2021, 51(4): 1414-1419.
[6] 金立生,郭柏苍,王芳荣,石健. 基于改进YOLOv3的车辆前方动态多目标检测算法[J]. 吉林大学学报(工学版), 2021, 51(4): 1427-1436.
[7] 张家旭,王欣志,赵健,施正堂. 汽车高速换道避让路径规划及离散滑模跟踪控制[J]. 吉林大学学报(工学版), 2021, 51(3): 1081-1090.
[8] 袁哲明,袁鸿杰,言雨璇,李钎,刘双清,谭泗桥. 基于深度学习的轻量化田间昆虫识别及分类模型[J]. 吉林大学学报(工学版), 2021, 51(3): 1131-1139.
[9] 何仁,赵晓聪,杨奕彬,王建强. 基于驾驶人风险响应机制的人机共驾模型[J]. 吉林大学学报(工学版), 2021, 51(3): 799-809.
[10] 彭博,张媛媛,王玉婷,唐聚,谢济铭. 基于自动编码机-分类器的视频交通状态自动识别[J]. 吉林大学学报(工学版), 2021, 51(3): 886-892.
[11] 宋大凤,杨丽丽,曾小华,王星琦,梁伟智,杨南南. 基于行驶工况合成的混合动力汽车电池寿命优化[J]. 吉林大学学报(工学版), 2021, 51(3): 781-791.
[12] 宋震,李俊良,刘贵强. 基于深度学习和限幅模糊的变转速液压动力源恒流量预测方法[J]. 吉林大学学报(工学版), 2021, 51(3): 1106-1110.
[13] 李锦青,周健,底晓强. 基于循环生成对抗网络的学习型光学图像加密方案[J]. 吉林大学学报(工学版), 2021, 51(3): 1060-1066.
[14] 宋强,孙丹婷,章伟. 纯电动车机械式自动变速器换挡非线性建模及控制[J]. 吉林大学学报(工学版), 2021, 51(3): 810-819.
[15] 王波,何洋扬,聂冰冰,许述财,张金换. 底部爆炸条件下车内乘员的腹部损伤[J]. 吉林大学学报(工学版), 2021, 51(3): 792-798.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!