Journal of Jilin University(Engineering and Technology Edition) ›› 2024, Vol. 54 ›› Issue (4): 959-968.doi: 10.13229/j.cnki.jdxbgxb.20220638

Previous Articles     Next Articles

Driver distracted driving detection based on improved YOLOv5

Ren-xiang CHEN1(),Chao-chao HU1,Xiao-lin HU2(),Li-xia YANG3,Jun ZHANG1,Jia-le HE1   

  1. 1.Chongqing Engineering Laboratory for Transportation Engineering Application Robot,Chongqing Jiaotong University,Chongqing 400074,China
    2.Chongqing Innovation Center of Industrial Big-Data Co. Ltd,Chongqing 400056,China
    3.College of Business and Management,Chongqing University of Science & Technology,Chongqing 401331,China
  • Received:2022-05-24 Online:2024-04-01 Published:2024-05-17
  • Contact: Xiao-lin HU E-mail:manlou.yue@126.com;huxl0918@163.com

Abstract:

To address the problem that distracted driving detection using classification methods can only identify a limited number of distracted driving behavior categories and ignore temporal information, we propose a distracted driving detection method based on improved YOLOv5. First, the Ghost module is introduced on the basis of YOLOv5, and linear transformation is used instead of partial conventional convolution for feature extraction to lighten the network model to achieve fast and accurate detection of cell phone, water bottle, driver's eyes and head region in the image; second, after obtaining the target detection results, the logic algorithm is designed to detect the presence of distracted driving in each frame by combining with head pose estimation. Second, on the basis of obtaining the target detection results, a logic algorithm is designed and integrated into YOLOv5 with head pose estimation to detect the presence of distracted driving in each frame from both cognitive distraction and visual distraction perspectives, which avoids the problem that the classification method is limited by the number of distracted driving categories, and then setting an appropriate time threshold, thus realizing real-time and effective distracted driving detection; finally, three sets of experiments are conducted on the collected driving behavior dataset of 18 drivers to verify the feasibility and effectiveness of the method in this paper.

Key words: distracted driving, YOLOv5, driving behavior, target detection, head pose estimation

CLC Number: 

  • U461.91

Fig.1

Focus slice operation"

Fig.2

Ghost module"

Fig.3

Structure diagram of YOLOv5-G"

Fig.4

Object detection obtains the area of driver′seyes and head"

Fig.5

Different head poses"

Fig.6

Flow chart of proposed method"

Fig.7

Part of image dataset"

Table 1

Comparison of experimental results of different target detection algorithms"

目标检测算法AP/%↑mAP/%↑Params/M↓FLOPs/G↓FPS↑
水杯手机左眼右眼头部
YOLOv5n97.166.095.995.499.490.761.694.2102.0
YOLOv5s98.589.498.599.399.397.006.7115.984.0
YOLOv5m98.380.095.398.199.394.2020.2149.164.9
SSD97.670.297.098.698.992.4625.12138.223.9
YOLOv395.367.896.398.799.091.4261.5577.632.7
YOLOv492.566.292.994.699.589.145.7816.483.2
YOLOv5-G97.486.298.999.399.396.224.8610.793.9

Table 2

Image dataset partitioning"

驾驶行为图像数/张
训练集测试集
正常驾驶984422
分心驾驶打电话554231
喝水293131
交谈223110
其他/568

Table 3

Comparison of experimental results on image datasets"

方法模型正常驾驶测试准确率/%↑分心驾驶测试准确率/%↑F1-Score↑
打电话喝水交谈其他合计
ResNet5095.7396.5497.7194.5526.9458.460.64
Mobilenet-v296.2110099.2493.6427.1170.040.65
Inception-v391.4710010098.1839.6173.940.67
本文方法96.68////95.580.93

Fig.8

Part of detection results of proposed method"

Fig.9

Call"

Fig.10

Drink"

Fig.11

Talk to passengers"

Fig.12

Normal driving"

1 Nee M, Contrand B, Orriols L, et al. Road safety and distraction, results from a responsibility case-control study among a sample of road users interviewed at the emergency room[J]. Accident Analysis and Prevention, 2019, 122: 19-24.
2 Craye C, Rashwan A, Kamel M S, et al. A multi-modal driver fatigue and distraction assessment system[J]. International Journal of Intelligent Transportation Systems Research, 2016, 14(3): 1-22.
3 白中浩, 王韫宇, 张林伟. 基于图卷积网络的多信息融合驾驶员分心行为检测[J]. 汽车工程, 2020, 313(8): 1027-1033.
Bai Zhong-hao, Wang Yun-yu, Zhang Lin-wei. Driver distraction behavior detection with multi-information fusion based on graph convolution networks [J]. Automotive Engineering, 2020, 313(8): 1027-1033.
4 秦斌斌, 彭良康, 卢向明, 等. 司机分心驾驶检测研究进展[J]. 计算机应用, 2021, 372(8): 2330-2337.
Qin Bin-bin, Peng Liang-kang, Lu Xiang-ming, et al. Research progress on driver distracted driving detection[J]. Journal of Computer Applications, 2021, 372(8): 2330-2337.
5 Zhao Z, Xia S, Xu X, et al. Driver distraction detection method based on continuous head pose estimation [J]. Computational Intelligence and Neuroscience, 2020, 4:1-10.
6 张兰芳, 崔博宇, 王俊骅, 等. 自然驾驶状态下使用手机对驾驶控制行为的影响[J]. 同济大学学报:自然科学版, 2019, 47(12): 1756-1763.
Zhang Lan-fang, Cui Bo-yu, Wang Jun-hua, et al. Effects of naturalistic mobile phone operations on driving control behavior[J]. Journal of Tongji University (Natural Science), 2019, 47(12): 1756-1763.
7 华强, 金立生, 郭柏苍, 等. 一种混行环境下驾驶人认知分心识别方法[J]. 吉林大学学报: 工学版,2022,52(8): 1800-1807.
Hua Qiang, Jin Li-sheng, Guo Bai-cang, et al. A recognition method for driver's cognitive distraction in simulated mixed traffic environment[J]. Journal of Jilin University(Engineering and Technology Edition), 2022,52(8): 1800-1807.
8 Oikawa H, Nishida T, Sakamoto R, et al. Fast semi-supervised anomaly detection of drivers' behavior using online sequential extreme learning machine[C]∥IEEE 23rd International Conference on Intelligent Transportation Systems(ITSC), Hangzhou, China, 2020: 1-8.
9 Guo Z Z, Pan Y F, Zhao G Z, et al. Detection of driver vigilance level using EEG signals and driving contexts[J]. IEEE Transactions on Reliability, 2018, 67(1): 370-280.
10 关伟, 杨柳, 江世雄, 等. 脑电在交通驾驶行为中的应用研究综述[J]. 交通运输系统工程与信息, 2016, 16(3): 35-44.
Guan Wei, Yang Liu, Jiang Shi-xiong, et al. A review of researches on the application of EEG in traffic driving behavior[J]. Journal of Transportation Systems Engineering and Information Technology, 2016, 16(3): 35-44.
11 Qin B B, Qian J B, Xin Y, et al. Distracted driver detection based on a CNN with decreasing filter size [J]. IEEE Transactions on Intelligent Transportation Systems, 2022, 23(7): 6922-6933.
12 Tang M, Wu F, Zhao L L, et al. Detection of distracted driving based on multi-granularity and middle-level features[C]∥Chinese Automation Congress (CAC), Shanghai, China, 2020: 2717-2722.
13 Huang C, Wang X C, Cao J N, et al. HCF: A hybrid CNN framework for behavior detection of distracted drivers[J]. IEEE Access, 2020, 99: 109335-109349.
14 Abouelnaga Y, Eraqi H M, Moustafa M N. Real-time distracted driver posture classification[C]∥Workshop on Machine Learning for Intelligent Transportation Systems, Montreal, Canda, 2018: 1-8.
15 StateFarm, State farm distracted driver detection dataset [EB/OL]. [2022-05-21]. .
16 尹智帅, 钟恕, 聂琳真, 等. 基于人体姿态估计的分心驾驶行为检测[J]. 中国公路学报, 2022, 35(6): 312-323.
Yin Zhi-shuai, Zhong Shu, Nie Lin-zhen, et al. Distracted driving behavior detection based on human pose estimation[J]. China Journal of Highway and Transport, 2022, 356(6): 312-323.
17 张斌, 付俊怡, 夏金祥. 基于类间距优化的分心驾驶行为识别模型训练方法[J]. 汽车工程, 2022, 44(2): 225-232.
Zhang Bin, Fu Jun-yi, Xia Jin-xiang. A metric space optimized method for driver distraction recognition model training[J]. Automotive Engineering, 2022, 44(2): 225-232.
18 Dey A K, Goel B, Chellappan S. Context-driven detection of distracted driving using images from in-car cameras[J]. Internet of Things, 2021, 14(6): 100380-100391.
19 Okan K, Zheng J P, Xu H, et al. Driver anomaly detection: a dataset and contrastive learning approach [C]∥IEEE Winter Conference on Application of Computer Vision, Online, 2021: 91-100.
20 Redmon J, Farhadi A. YOLOv3: an incremental improvement [EB/OL]. [2022-05-21]. .
21 Bochkovskiy A, Wang C Y, Liao H Y M, et al. YOLOv4: Optimal speed and accuracy of object detection [EB/OL]. [2022-05-21]. .
22 Han K, Wang Y, Tian Q, et al. GhostNet: More features from cheap operations[C]∥IEEE/CVF Conference on Computer Vision and Pattern Recognition, Online 2020: 1-10.
[1] Hong ZHANG,Zhi-wei ZHU,Tian-yu HU,Yan-feng GONG,Jian-ting ZHOU. Bridge bolt defect identification method based on improved YOLOv5s [J]. Journal of Jilin University(Engineering and Technology Edition), 2024, 54(3): 749-760.
[2] Hong-zhi WANG,Ming-xuan SONG,Chao CHENG,Dong-xuan XIE. Vehicle distance warning method based on improved YOLOv4⁃tiny algorithm [J]. Journal of Jilin University(Engineering and Technology Edition), 2024, 54(3): 741-748.
[3] Ya-li ZHANG,Rui FU,Wei YUAN,Ying-shi GUO. Classification and recognition model of entering and leaving stops' driving style considering energy consumption [J]. Journal of Jilin University(Engineering and Technology Edition), 2023, 53(7): 2029-2042.
[4] Can-can SONG,Di-fei JING,Jun-feng XIE,Ke-xin KANG. Analysis on driving behaviors on flat curved sections of highways with advertising signs [J]. Journal of Jilin University(Engineering and Technology Edition), 2023, 53(5): 1345-1354.
[5] Shan XUE,Ya-liang ZHANG,Qiong-ying LYU,Guo-hua CAO. Anti⁃unmanned aerial vehicle system object detection algorithm under complex background [J]. Journal of Jilin University(Engineering and Technology Edition), 2023, 53(3): 891-901.
[6] Jin XU,Yan-peng WANG,Hai-yuan CHEN,Xiao-bo ZHANG,Cun-shu PAN. Longitudinal driving characteristics and operating speed prediction model of cars on hairpin curves of mountainous roads [J]. Journal of Jilin University(Engineering and Technology Edition), 2023, 53(12): 3432-3445.
[7] Qiang HUA,Li-sheng JIN,Bai-cang GUO,Shun-ran ZHANG,Yu-han WANG. A recognition method for driver's cognitive distraction in simulated mixed traffic environment [J]. Journal of Jilin University(Engineering and Technology Edition), 2022, 52(8): 1800-1807.
[8] Ming-hua GAO,Can YANG. Traffic target detection method based on improved convolution neural network [J]. Journal of Jilin University(Engineering and Technology Edition), 2022, 52(6): 1353-1361.
[9] Shao-bo JI,Yang LI,Meng LI,Shi-bin SU,Xiao-long MA,Shao-qing HE,Guo-rui JIA,Yong CHENG. Influence of driving behavior on energy consumption of pure electric shared vehicles [J]. Journal of Jilin University(Engineering and Technology Edition), 2022, 52(4): 754-763.
[10] Xue-yun CHEN,Tao XU,Xiao-qiao HUANG. Detection method of medical cell image generation based on conditional generative adversarial network [J]. Journal of Jilin University(Engineering and Technology Edition), 2021, 51(4): 1414-1419.
[11] Jin XU,Cun-shu PAN,Jing-hou FU,Jun LIU,Dan-qi WANG. Speed behavior characteristic on typical driving scenarios and along switched scenarios [J]. Journal of Jilin University(Engineering and Technology Edition), 2021, 51(4): 1331-1341.
[12] De-lun PAN,Jun JI,Yue-jin ZHANG. Dynamic object detection method of video surveillance based on motion vector space coding [J]. Journal of Jilin University(Engineering and Technology Edition), 2021, 51(4): 1370-1374.
[13] Ping WAN,Chao-zhong WU,Xiao-feng MA. Discriminating threshold of driving anger intensity based on driving behavior features by ROC curve analysis [J]. Journal of Jilin University(Engineering and Technology Edition), 2020, 50(1): 121-131.
[14] Liang JIANG,Yi HE. Risky driving behavior and influencing factors analysis for electric two⁃wheeler [J]. Journal of Jilin University(Engineering and Technology Edition), 2019, 49(4): 1107-1113.
[15] WAN Ping, WU Chao-zhong, LIN Ying-zi, MA Xiao-feng. Driving anger detection based on multivariate time series features of driving behavior [J]. 吉林大学学报(工学版), 2017, 47(5): 1426-1435.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!