基于改进YOLOv5的驾驶员分心驾驶检测

doi:10.13229/j.cnki.jdxbgxb.20220638

吉林大学学报(工学版) ›› 2024, Vol. 54 ›› Issue (4): 959-968.doi: 10.13229/j.cnki.jdxbgxb.20220638

• 交通运输工程·土木工程 • 上一篇下一篇

基于改进YOLOv5的驾驶员分心驾驶检测

陈仁祥¹(),胡超超¹,胡小林²(),杨黎霞³,张军¹,何家乐¹

^1.重庆交通大学交通工程应用机器人重庆市工程实验室，重庆 400074
^2.重庆工业大数据创新中心有限公司，重庆 400056
^3.重庆科技学院工商管理学院，重庆 401331

收稿日期:2022-05-24 出版日期:2024-04-01 发布日期:2024-05-17
通讯作者: 胡小林 E-mail:manlou.yue@126.com;huxl0918@163.com
作者简介:陈仁祥（1983-），男，教授，博士.研究方向：智能测控技术.E-mail： manlou.yue@126.com
基金资助:
国家重点研发计划项目(2018YFB1306601);国家自然科学基金项目(51975079);重庆市自然科学基金创新发展联合基金项目(CSTB2023NSCQ-LZX0127);重庆市研究生联合培养基地项目(JDLHPYJD2021007);重庆市教育委员会科学技术研究项目(KJZD-M202200701);重庆交通大学研究生科研创新项目(2022S0045)

Driver distracted driving detection based on improved YOLOv5

Ren-xiang CHEN¹(),Chao-chao HU¹,Xiao-lin HU²(),Li-xia YANG³,Jun ZHANG¹,Jia-le HE¹

^1.Chongqing Engineering Laboratory for Transportation Engineering Application Robot，Chongqing Jiaotong University，Chongqing 400074，China
^2.Chongqing Innovation Center of Industrial Big-Data Co. Ltd，Chongqing 400056，China
^3.College of Business and Management，Chongqing University of Science ＆ Technology，Chongqing 401331，China

Received:2022-05-24 Online:2024-04-01 Published:2024-05-17
Contact: Xiao-lin HU E-mail:manlou.yue@126.com;huxl0918@163.com

摘要/Abstract

摘要：

针对采用分类方法进行分心驾驶检测存在只能识别有限分心驾驶行为类别以及忽视时间信息的问题，提出了基于改进YOLOv5的驾驶员分心驾驶检测方法。首先，在YOLOv5的基础上引入Ghost模块，采用线性变换代替部分常规卷积进行特征提取以轻量化网络模型，实现快速又准确地检测图像中手机、水杯、驾驶员双眼和头部区域；其次，在获取目标检测结果的基础上，结合头部姿态估计设计逻辑算法并融入YOLOv5中，从认知分心和视觉分心两个角度检测每帧图像中驾驶员是否存在分心驾驶，避免了分类方法受限分心驾驶类别数的问题，再设置适当的时间阈值，从而实现端到端实时的分心驾驶预警；最后，对采集的18名驾驶员的驾驶行为数据集进行对比试验，验证了本文方法的可行性和有效性。

关键词: 分心驾驶, YOLOv5, 驾驶行为, 目标检测, 头部姿态估计

Abstract:

To address the problem that distracted driving detection using classification methods can only identify a limited number of distracted driving behavior categories and ignore temporal information， we propose a distracted driving detection method based on improved YOLOv5. First， the Ghost module is introduced on the basis of YOLOv5， and linear transformation is used instead of partial conventional convolution for feature extraction to lighten the network model to achieve fast and accurate detection of cell phone， water bottle， driver's eyes and head region in the image； second， after obtaining the target detection results， the logic algorithm is designed to detect the presence of distracted driving in each frame by combining with head pose estimation. Second， on the basis of obtaining the target detection results， a logic algorithm is designed and integrated into YOLOv5 with head pose estimation to detect the presence of distracted driving in each frame from both cognitive distraction and visual distraction perspectives， which avoids the problem that the classification method is limited by the number of distracted driving categories， and then setting an appropriate time threshold， thus realizing real-time and effective distracted driving detection； finally， three sets of experiments are conducted on the collected driving behavior dataset of 18 drivers to verify the feasibility and effectiveness of the method in this paper.

Key words: distracted driving, YOLOv5, driving behavior, target detection, head pose estimation

中图分类号:

U461.91

陈仁祥,胡超超,胡小林,杨黎霞,张军,何家乐. 基于改进YOLOv5的驾驶员分心驾驶检测[J]. 吉林大学学报(工学版), 2024, 54(4): 959-968.

Ren-xiang CHEN,Chao-chao HU,Xiao-lin HU,Li-xia YANG,Jun ZHANG,Jia-le HE. Driver distracted driving detection based on improved YOLOv5[J]. Journal of Jilin University(Engineering and Technology Edition), 2024, 54(4): 959-968.

图/表 15

图1

图2

图3

图4

图5

图6

图7

表1

表2

表3

图8

图9

图10

图11

图12

参考文献 22

1	Nee M, Contrand B, Orriols L, et al. Road safety and distraction, results from a responsibility case-control study among a sample of road users interviewed at the emergency room[J]. Accident Analysis and Prevention, 2019, 122: 19-24.
2	Craye C, Rashwan A, Kamel M S, et al. A multi-modal driver fatigue and distraction assessment system[J]. International Journal of Intelligent Transportation Systems Research, 2016, 14(3): 1-22.
3	白中浩, 王韫宇, 张林伟. 基于图卷积网络的多信息融合驾驶员分心行为检测[J]. 汽车工程, 2020, 313(8): 1027-1033.
	Bai Zhong-hao, Wang Yun-yu, Zhang Lin-wei. Driver distraction behavior detection with multi-information fusion based on graph convolution networks [J]. Automotive Engineering, 2020, 313(8): 1027-1033.
4	秦斌斌, 彭良康, 卢向明, 等. 司机分心驾驶检测研究进展[J]. 计算机应用, 2021, 372(8): 2330-2337.
	Qin Bin-bin, Peng Liang-kang, Lu Xiang-ming, et al. Research progress on driver distracted driving detection[J]. Journal of Computer Applications, 2021, 372(8): 2330-2337.
5	Zhao Z, Xia S, Xu X, et al. Driver distraction detection method based on continuous head pose estimation [J]. Computational Intelligence and Neuroscience, 2020, 4:1-10.
6	张兰芳, 崔博宇, 王俊骅, 等. 自然驾驶状态下使用手机对驾驶控制行为的影响[J]. 同济大学学报:自然科学版, 2019, 47(12): 1756-1763.
	Zhang Lan-fang, Cui Bo-yu, Wang Jun-hua, et al. Effects of naturalistic mobile phone operations on driving control behavior[J]. Journal of Tongji University (Natural Science), 2019, 47(12): 1756-1763.
7	华强, 金立生, 郭柏苍, 等. 一种混行环境下驾驶人认知分心识别方法[J]. 吉林大学学报: 工学版,2022,52(8): 1800-1807.
	Hua Qiang, Jin Li-sheng, Guo Bai-cang, et al. A recognition method for driver's cognitive distraction in simulated mixed traffic environment[J]. Journal of Jilin University(Engineering and Technology Edition), 2022,52(8): 1800-1807.
8	Oikawa H, Nishida T, Sakamoto R, et al. Fast semi-supervised anomaly detection of drivers' behavior using online sequential extreme learning machine[C]∥IEEE 23rd International Conference on Intelligent Transportation Systems(ITSC), Hangzhou, China, 2020: 1-8.
9	Guo Z Z, Pan Y F, Zhao G Z, et al. Detection of driver vigilance level using EEG signals and driving contexts[J]. IEEE Transactions on Reliability, 2018, 67(1): 370-280.
10	关伟, 杨柳, 江世雄, 等. 脑电在交通驾驶行为中的应用研究综述[J]. 交通运输系统工程与信息, 2016, 16(3): 35-44.
	Guan Wei, Yang Liu, Jiang Shi-xiong, et al. A review of researches on the application of EEG in traffic driving behavior[J]. Journal of Transportation Systems Engineering and Information Technology, 2016, 16(3): 35-44.
11	Qin B B, Qian J B, Xin Y, et al. Distracted driver detection based on a CNN with decreasing filter size [J]. IEEE Transactions on Intelligent Transportation Systems, 2022, 23(7): 6922-6933.
12	Tang M, Wu F, Zhao L L, et al. Detection of distracted driving based on multi-granularity and middle-level features[C]∥Chinese Automation Congress (CAC), Shanghai, China, 2020: 2717-2722.
13	Huang C, Wang X C, Cao J N, et al. HCF: A hybrid CNN framework for behavior detection of distracted drivers[J]. IEEE Access, 2020, 99: 109335-109349.
14	Abouelnaga Y, Eraqi H M, Moustafa M N. Real-time distracted driver posture classification[C]∥Workshop on Machine Learning for Intelligent Transportation Systems, Montreal, Canda, 2018: 1-8.
15	StateFarm, State farm distracted driver detection dataset [EB/OL]. [2022-05-21]. .
16	尹智帅, 钟恕, 聂琳真, 等. 基于人体姿态估计的分心驾驶行为检测[J]. 中国公路学报, 2022, 35(6): 312-323.
	Yin Zhi-shuai, Zhong Shu, Nie Lin-zhen, et al. Distracted driving behavior detection based on human pose estimation[J]. China Journal of Highway and Transport, 2022, 356(6): 312-323.
17	张斌, 付俊怡, 夏金祥. 基于类间距优化的分心驾驶行为识别模型训练方法[J]. 汽车工程, 2022, 44(2): 225-232.
	Zhang Bin, Fu Jun-yi, Xia Jin-xiang. A metric space optimized method for driver distraction recognition model training[J]. Automotive Engineering, 2022, 44(2): 225-232.
18	Dey A K, Goel B, Chellappan S. Context-driven detection of distracted driving using images from in-car cameras[J]. Internet of Things, 2021, 14(6): 100380-100391.
19	Okan K, Zheng J P, Xu H, et al. Driver anomaly detection: a dataset and contrastive learning approach [C]∥IEEE Winter Conference on Application of Computer Vision, Online, 2021: 91-100.
20	Redmon J, Farhadi A. YOLOv3: an incremental improvement [EB/OL]. [2022-05-21]. .
21	Bochkovskiy A, Wang C Y, Liao H Y M, et al. YOLOv4: Optimal speed and accuracy of object detection [EB/OL]. [2022-05-21]. .
22	Han K, Wang Y, Tian Q, et al. GhostNet: More features from cheap operations[C]∥IEEE/CVF Conference on Computer Vision and Pattern Recognition, Online 2020: 1-10.

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed

目标检测算法	AP/%↑					mAP/%↑	Params/M↓	FLOPs/G↓	FPS↑
目标检测算法	水杯	手机	左眼	右眼	头部	mAP/%↑	Params/M↓	FLOPs/G↓	FPS↑
YOLOv5n	97.1	66.0	95.9	95.4	99.4	90.76	1.69	4.2	102.0
YOLOv5s	98.5	89.4	98.5	99.3	99.3	97.00	6.71	15.9	84.0
YOLOv5m	98.3	80.0	95.3	98.1	99.3	94.20	20.21	49.1	64.9
SSD	97.6	70.2	97.0	98.6	98.9	92.46	25.12	138.2	23.9
YOLOv3	95.3	67.8	96.3	98.7	99.0	91.42	61.55	77.6	32.7
YOLOv4	92.5	66.2	92.9	94.6	99.5	89.14	5.78	16.4	83.2
YOLOv5-G	97.4	86.2	98.9	99.3	99.3	96.22	4.86	10.7	93.9

驾驶行为		图像数/张
驾驶行为		训练集	测试集
正常驾驶		984	422
分心驾驶	打电话	554	231
	喝水	293	131
	交谈	223	110
	其他	/	568

方法模型	正常驾驶测试准确率/%↑	分心驾驶测试准确率/%↑					F1-Score↑
方法模型	正常驾驶测试准确率/%↑	打电话	喝水	交谈	其他	合计	F1-Score↑
ResNet50	95.73	96.54	97.71	94.55	26.94	58.46	0.64
Mobilenet-v2	96.21	100	99.24	93.64	27.11	70.04	0.65
Inception-v3	91.47	100	100	98.18	39.61	73.94	0.67
本文方法	96.68	/	/	/	/	95.58	0.93

[1]	张云佐,郭威,李文博. 遥感图像密集小目标全方位精准检测算法[J]. 吉林大学学报(工学版), 2024, 54(4): 1105-1113.
[2]	张洪,朱志伟,胡天宇,龚燕峰,周建庭. 基于改进YOLOv5s的桥梁螺栓缺陷识别方法[J]. 吉林大学学报(工学版), 2024, 54(3): 749-760.
[3]	王宏志,宋明轩,程超,解东旋. 基于改进YOLOv4-tiny算法的车距预警方法[J]. 吉林大学学报(工学版), 2024, 54(3): 741-748.
[4]	王春华,李恩泽,肖敏. 多特征融合和孪生注意力网络的高分辨率遥感图像目标检测[J]. 吉林大学学报(工学版), 2024, 54(1): 240-250.
[5]	张雅丽,付锐,袁伟,郭应时. 考虑能耗的进出站驾驶风格分类及识别模型[J]. 吉林大学学报(工学版), 2023, 53(7): 2029-2042.
[6]	宋灿灿,荆迪菲,谢俊峰,康可心. 设置广告牌的高速公路平曲线路段驾驶行为分析[J]. 吉林大学学报(工学版), 2023, 53(5): 1345-1354.
[7]	陶博,颜伏伍,尹智帅,武冬梅. 基于高精度地图增强的三维目标检测算法[J]. 吉林大学学报(工学版), 2023, 53(3): 802-809.
[8]	薛珊,张亚亮,吕琼莹,曹国华. 复杂背景下的反无人机系统目标检测算法[J]. 吉林大学学报(工学版), 2023, 53(3): 891-901.
[9]	徐进,王延鹏,陈海源,张晓波,潘存书. 山区公路回头曲线小客车纵向行驶特性及运行速度模型[J]. 吉林大学学报(工学版), 2023, 53(12): 3432-3445.
[10]	华强,金立生,郭柏苍,张舜然,王禹涵. 一种混行环境下驾驶人认知分心识别方法[J]. 吉林大学学报(工学版), 2022, 52(8): 1800-1807.
[11]	高明华,杨璨. 基于改进卷积神经网络的交通目标检测方法[J]. 吉林大学学报(工学版), 2022, 52(6): 1353-1361.
[12]	纪少波,李洋,李萌,苏士斌,马晓龙,何绍清,贾国瑞,程勇. 纯电动共享汽车驾驶行为对能耗的影响[J]. 吉林大学学报(工学版), 2022, 52(4): 754-763.
[13]	曲优,李文辉. 基于锚框变换的单阶段旋转目标检测方法[J]. 吉林大学学报(工学版), 2022, 52(1): 162-173.
[14]	潘德伦,冀隽,张跃进. 基于运动矢量空间编码的视频监控动态目标检测方法[J]. 吉林大学学报(工学版), 2021, 51(4): 1370-1374.
[15]	金立生,郭柏苍,王芳荣,石健. 基于改进YOLOv3的车辆前方动态多目标检测算法[J]. 吉林大学学报(工学版), 2021, 51(4): 1427-1436.

基于改进YOLOv5的驾驶员分心驾驶检测

Driver distracted driving detection based on improved YOLOv5

RICH HTML

PDF (PC)

摘要/Abstract

引用本文

使用本文

图/表 15

参考文献 22

相关文章 15

Metrics

本文评价

推荐阅读 0