Journal of Jilin University(Engineering and Technology Edition) ›› 2021, Vol. 51 ›› Issue (4): 1437-1446.doi: 10.13229/j.cnki.jdxbgxb20200380

Previous Articles    

DG-SLAM algorithm for dynamic scene compound deep learning and parallel computing

Feng-chong LAN1,2(),Ji-wen LI1,2,Ji-qing CHEN1,2()   

  1. 1.School of Mechanical & Automotive Engineering，South China University of Technology，Guangzhou 510640，China
    2.Guangdong Key Laboratory of Automotive Engineering，South China University of Technology，Guangzhou 510640，China
  • Received:2020-05-30 Online:2021-07-01 Published:2021-07-14
  • Contact: Ji-qing CHEN E-mail:fclan@scut.edu.cn;chjq@scut.edu.cn

RICH HTML

 6   

PDF (PC)

118

Abstract:

In view of the disadvantages of the existing simultaneous localization and mapping (SLAM) algorithm, which has low real-time performance and the positioning accuracy is greatly reduced in dynamic environment, a DG-SLAM algorithm based on deep learning and GPU parallel computing was proposed. The deep learning-based object detection algorithm was introduced to detect dynamic objects in the driving environment, and the feature points of dynamic objects were removed before the matching of image frames, so as to eliminate the impact of mismatching dynamic objects on the positioning accuracy of SLAM system. In the tracking of local maps of ORB-SLAM2, the discriminant method of 3D interior points was used to distinguish the inner points and outer points, and the GPU parallel computing model was established to efficiently search the local map points. The Saturated kernel function was used to minimize the reprojection error of the two norm terms to ensure the parallel calculation of the reprojection error when the local map was optimizing. The algorithm was verified on the KITII dataset, DG-SLAM has high tracking accuracy, and the average calculation efficiency was more than 3.4 times faster than that of ORB-SLAM2 system under the same conditions, more than 85 frames per second, which could realize efficient and high precision SLAM in dynamic scene.

Key words: vehicle engineering, simultaneous localization and mapping, deep learning, object detection, parallel computing

CLC Number: 

  • TP391.4

Fig.1

Overall system architecture of DG-SLAM"

Fig.2

ORB-SLAM2 positioning process"

Fig.3

Parallel computing process for localization of DG-SLAM"

Fig. 4

Variation trend of saturated kernel function under different residual thresholds"

Fig.5

SSD network structure"

Fig.6

Data association comparison diagram before and after removing dynamic feature points"

Fig.7

KITTI dataset road types"

Fig.8

Comparison of position estimation between DG-SLAM and ORB-SLAM2 on KITTI 01 dataset"

Fig.9

Comparison of attitude angle estimation between DG-SLAM and ORB-SLAM2 on KITTI 01 dataset"

Fig.10

Trajectory error of ORB-SLAM2 tested on KITTI 01 dataset"

Fig.11

Trajectory error of DG-SLAM tested on KITTI 01 dataset"

Fig.12

Absolute pose errors of DG-SLAM and ORB-SLAM2"

Table 1

Test accuracy on KITTI dataset"

序列编号场景类型轨迹长度/mRMSE
ORB?SLAM2DG?SLAM
平均值--20.73717.692
00城市支路714.269.6169.560
01高速公路2453.2011.7461.666
02城市主干路5067.2347.15028.354
03城市主干路560.880.9200.918
04城市快速路393.640.7030.423
05城市支路2205.5744.32444.532
06城市支路1232.8729.34927.258
07城市支路694.6917.87316.828
08城市支路3222.7955.18452.689
09城市主干路1705.0558.73349.568
10城市支路919.516.3846.356

Fig.13

Absolute pose errors of DG-SLAM and ORB-SLAM2 on KITTI 04"

Fig.14

Absolute pose errors of DG-SLAM and ORB-SLAM2 on KITTI 02"

Fig.15

Absolute pose errors of DG-SLAM and ORB-SLAM2 on KITTI 03"

Fig.16

Comparison of time consuming on KITTI 01 dataset between DG-SLAM and ORB-SLAM2"

1 Yousif K, Bab-Hadiashar A, Hoseinnezhad R. An overview to visual odometry and visual SLAM: applications to mobile robotics[J]. Intelligent Industrial Systems, 2015, 1(4): 289-311.
2 邸凯昌,万文辉,赵红颖,等.视觉SLAM技术的进展与应用[J].测绘学报,2018,47(6):770-779.
Di Kai-chang, Wan Wen-hui, Zhao Hong-ying, et al. Progress and applications of visual SLAM[J]. Acta Geodaetica et Cartographica Sinica, 2018,47(6):770-779.
3 高成强,张云洲,王晓哲, 等. 面向室内动态环境的半直接法RGB-D SLAM算法[J].机器人, 2019, 41(3): 372-383.
Gao Cheng-qiang, Zhang Yun-zhou, Wang Xiao-Zhe, et al. Semi-direct RGB-D SLAM algorithm for dynamic indoor environments[J]. Robot, 2019, 41(3): 372-383.
4 Newcombe R A, Fox D, Seitz S M. Dynamic fusion: reconstruction and tracking of non-rigid scenes in real-time[C]∥The IEEE Conference on Computer Vision and Pattern Recognition(CVPR), Portland, Oregon, 2015: 343-352
5 Saputra M R U, Markham A, Trigoni N. Visual SLAM andstructure from motion in dynamic environments[J]. ACM Computing Surveys, 2018, 51(2): 1-36.
6 Klein G, Murray D. Parallel tracking and mapping for small AR workspaces[C]∥The 6th IEEE and ACM International Symposium on Mixed and Augmented Reality, Arlington, USA, 2007: 225-234.
7 Zhou Yong-long, Ma Kui-zhi, Jiang Xiang, et al. Parallelization and optimization of sift on gpu using cuda[C]∥The IEEE 10th International Conference on High Performance Computing and Communications & IEEE International Conference on Embedded and Ubiquitous Computing, Dalian, China, 2008: 1351-1358.
8 Bescos B, Fácil J M, Civera J, et al. DynaSLAM: Tracking, mapping, and inpainting in dynamic scenes[J]. IEEE Robotics and Automation Letters, 2018, 3(4): 4076-4083.
9 Kawewong A, Tongprasit N, Tangruamsub S, et al. Online and incremental appearance-based SLAM in highly dynamic environments[J]. International Journal of Robotics Research, 2011, 30(1):33-55.
10 Zhong F, Wang S, Zhang Z, et al. Detect-slam: Making object detection and slam mutually beneficial[C]∥IEEE Winter Conference on Applications of Computer Vision (WACV), Lake Tahoe, USA, 2018: 1001-1010.
11 Hosseinyalamdary S. Deep Kalman filter: simultaneous multi-sensor integration and modelling; a GNSS/IMU case study[J]. Sensors, 2018, 18(5): 1316.
12 Mur-Artal R, Tardós J D. ORB-SLAM2: an open-source slam system for monocular, stereo, and RGB-D cameras[J]. IEEE Transactions on Robotics, 2017, 33(5): 1255-1262.
13 Mur-Artal R, Montiel J M M, Tardos J D. ORB-SLAM: a versatile and accurate monocular SLAM system[J]. IEEE Transactions on Robotics, 2015, 31(5): 1147-1163.
14 李涛, 董前琨, 张帅, 等. 基于线程池的GPU任务并行计算模式研究[J]. 计算机学报, 2018, 41(10): 2175-2192.
Li Tao, Dong Qian-kun, Zang Shuai, et al. GPU task parallel computing paradigm based on thread pool model[J]. Chinese Journal of Computers, 2018, 41(10): 2175-2192.
15 Rublee E, Rabaud V, Konolige K, et al. ORB: an efficient alternative to SIFT or SURF[C]∥The International Conference on Computer Vision, Barcelona, Spain, 2011: 2564-2571.
16 Bay H, Tuytelaars T, van Gool L. Surf: speeded up robust features[C]∥European Conference on Computer Vision, Berlin, Germany, 2006: 404-417.
17 Lowe D G. Distinctive image features from scale-invariant keypoints[J]. International Journal of Computer Vision, 2004, 60(2): 91-110.
18 Lepetit V, Moreno-Noguer F, Fua P. EPnP: an accurate on solution to the PnP problem[J]. International Journal of Computer Vision, 2009, 81(2): 155-166.
19 Liu W, Anguelov D, Erhan D, et al. SSD: single shot multibox detector[C]∥European Conference on Computer Vision, Amsterdam, The Netherlands, 2016: 21-37.
20 Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition [J]. arXiv preprint arXiv:, 2014.
21 Geiger A, Lenz P, Stiller C, et al. Vision meets robotics: the KITTI dataset[J]. The International Journal of Robotics Research, 2013, 32(11): 1231-1237.
[1] Xin CHEN,Gui-shen YU,Biao ZHANG,Kai-xuan PAN,Li-fei YANG. Equivalent modeling of tensile-shear behavior for friction stir spot welding joints [J]. Journal of Jilin University(Engineering and Technology Edition), 2021, 51(4): 1190-1197.
[2] Jian YANG,Qi XIA,Hai-chao ZHOU,Guo-lin WANG. Noise reduction mechanism of truck radial tire based on modified carcass string contour design [J]. Journal of Jilin University(Engineering and Technology Edition), 2021, 51(4): 1198-1203.
[3] Jiang-qi LONG,Jin-tao XIANG,Ping YU,Jun-cheng WANG. Linear disturbance observer suitable for sliding mode control of nonlinear active suspension [J]. Journal of Jilin University(Engineering and Technology Edition), 2021, 51(4): 1230-1240.
[4] Li-sheng JIN,Bai-cang GUO,Fang-rong WANG,Jian SHI. Dynamic multiple object detection algorithm for vehicle forward based on improved YOLOv3 [J]. Journal of Jilin University(Engineering and Technology Edition), 2021, 51(4): 1427-1436.
[5] Jia-xu ZHANG,Xin-zhi WANG,Jian ZHAO,Zheng-tang SHI. Path planning and discrete sliding mode tracking control for high⁃speed lane changing collision avoidance of vehicle [J]. Journal of Jilin University(Engineering and Technology Edition), 2021, 51(3): 1081-1090.
[6] Zhe-ming YUAN,Hong-jie YUAN,Yu-xuan YAN,Qian LI,Shuang-qing LIU,Si-qiao TAN. Automatic recognition and classification of field insects based on lightweight deep learning model [J]. Journal of Jilin University(Engineering and Technology Edition), 2021, 51(3): 1131-1139.
[7] Ren HE,Xiao-cong ZHAO,Yi-bin YANG,Jian-qiang WANG. Man⁃machine shared driving model using risk⁃response mechanism of human driver [J]. Journal of Jilin University(Engineering and Technology Edition), 2021, 51(3): 799-809.
[8] Bo PENG,Yuan-yuan ZHANG,Yu-ting WANG,Ju TANG,Ji-ming XIE. Automatic traffic state recognition from videos based on auto⁃encoder and classifiers [J]. Journal of Jilin University(Engineering and Technology Edition), 2021, 51(3): 886-892.
[9] Da-feng SONG,Li-li YANG,Xiao-hua ZENG,Xing-qi WANG,Wei-zhi LIANG,Nan-nan YANG. Battery life optimization of hybrid electric vehicle based on driving cycle construction [J]. Journal of Jilin University(Engineering and Technology Edition), 2021, 51(3): 781-791.
[10] Zhen SONG,Jun-liang LI,Gui-qiang LIU. Constant flow prediction method of variable speed hydraulic power source based on deep learning and limitation fuzzy [J]. Journal of Jilin University(Engineering and Technology Edition), 2021, 51(3): 1106-1110.
[11] Jin-qing LI,Jian ZHOU,Xiao-qiang DI. Learning optical image encryption scheme based on CycleGAN [J]. Journal of Jilin University(Engineering and Technology Edition), 2021, 51(3): 1060-1066.
[12] Qiang SONG,Dan-ting SUN,Wei ZHANG. Shift nonlinear modeling and control of automated mechanical transmission in pure electric vehicle [J]. Journal of Jilin University(Engineering and Technology Edition), 2021, 51(3): 810-819.
[13] Bo WANG,Yang-yang HE,Bing-bing NIE,Shu-cai XU,Jin-huan ZHANG. Abdominal injury of vehicle occupant in underbody blast events [J]. Journal of Jilin University(Engineering and Technology Edition), 2021, 51(3): 792-798.
[14] Wei-min ZHUANG,Peng-yue WANG,Rui-juan GAO,Dong-xuan XIE. Effect of hot forming on static mechanical properties of AA5754 aluminum alloy [J]. Journal of Jilin University(Engineering and Technology Edition), 2021, 51(3): 847-854.
[15] Shou-tao LI,Rui WANG,Jing-chun XU,De-jun WANG,Yan-tao TIAN,Ding-li YU. A vehicle collision avoidance control method based on model predictive composite control [J]. Journal of Jilin University(Engineering and Technology Edition), 2021, 51(2): 738-746.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
Copyright © Journal of Jilin University(Engineering and Technology Edition), All Rights Reserved.
Tel: +86-431-85095297　Fax: +86-431-85094074　E-mail: xbgxb@jlu.edu.cn
Powered by Beijing Magtech Co. Ltd