基于双分支和可变形卷积网络的驾驶员行为识别方法

doi:10.13229/j.cnki.jdxbgxb.20230313

摘要/Abstract

摘要：

针对汽车座舱内的驾驶员行为识别任务，本文提出了一种基于双分支神经网络的识别方法。网络模型的主分支以ResNet 50作为主干网络进行特征提取，利用可变形卷积使模型适应驾驶员在图像中的形状和位置变化。辅助分支在梯度反向传播过程中辅助更新主干网络的参数，使主干网络能够更好地提取有利于驾驶员行为识别的特征，从而提高模型的识别性能。网络模型在State Farm 公开数据集的消融实验和对比实验结果表明：本文网络模型的识别准确率可以达到96.23%，针对易于混淆的行为类别识别效果更佳。研究结果对于汽车座舱内的驾驶员行为理解与保障行车安全具有重要意义。

关键词: 车辆工程, 智能驾驶, 驾驶员行为识别, 卷积神经网络, 辅助分支, 可变形卷积

Abstract:

This research offers a recognition approach based on a dual-branch neural network for recognizing driver behavior in the vehicle cockpit. The main branch of the network model employs ResNet 50 as the backbone network for feature extraction， and employs deformable convolution to adapt the model to changes in the shape and position of the driver in the image. The auxiliary branch aids in updating the parameters of the backbone network during the gradient backpropagation process， so that the backbone network can better extract features that are beneficial to driver behavior recognition， thereby improving the recognition performance. The results of ablation experiments and comparing experiments of the network model on the State Farm public dataset reveal that the proposed network model has a recognition accuracy of 96.23% and a better recognition effect on easily confused behavior categories. The study's findings are critical for understanding driver behavior in the vehicle cockpit and guaranteeing driving safety.

Key words: vehicle engineering, intelligent driving, driver behavior recognition, convolution neural network, auxiliary branch, deformable convolution

中图分类号:

U471.3

胡宏宇,张争光,曲优,蔡沐雨,高菲,高镇海. 基于双分支和可变形卷积网络的驾驶员行为识别方法[J]. 吉林大学学报(工学版), 2025, 55(1): 93-104.

Hong-yu HU,Zheng-guang ZHANG,You QU,Mu-yu CAI,Fei GAO,Zhen-hai GAO. Driver behavior recognition method based on dual-branch and deformable convolutional neural networks[J]. Journal of Jilin University(Engineering and Technology Edition), 2025, 55(1): 93-104.

图/表 11

图1

图2

图3

表 1

选取不同损失权重系数时各类行为的识别精确率及测试集总体准确率 (%)"

权重系数 $α$	C0	C1	C2	C3	C4	C5	C6	C7	C8	C9	总体准确率
0.5	82.21	99.55	99.79	99.36	99.15	99.58	98.95	100.0	91.51	85.2	95.74
0.6	84.66	99.55	99.79	99.15	98.52	100.0	99.37	99.42	92.00	88.16	96.23
0.7	82.38	99.55	99.79	99.15	98.73	99.58	99.79	99.71	81.71	82.86	94.81
0.8	80.00	99.33	99.58	99.57	99.79	100.0	98.48	99.71	83.85	87.13	94.98
0.9	84.17	99.78	99.58	99.36	98.94	99.37	99.58	100.0	86.31	75.47	94.60

表 1

图4

表 2

图5

图6

表 3

图7

表 4

参考文献 23

1	Dogan E, Rahal M C, Deborne R, et al. Transition of control in a partially automated vehicle: effects of anticipation and non-driving-related task involvement [J]. Transportation Research Part F: Traffic Psychology and Behaviour, 2017, 46(2017): 205-215.
2	Deeb J, Zakaria F, Kamali W, et al. Method of detection of early falling asleep while driving using EOG analysis[C]∥Proceedings of the 4th International Conference on Advances in Biomedical Engineering (ICABME), Beirut, Lebanon, 2017:150-153.
3	Halomoan J, Ramli K, Sudiana D. Statistical analysis to determine the ground truth of fatigue driving state using ECG recording and subjective reporting[C]∥Proceedings of the 1st International Conference on Information Technology, Advanced Mechanical and Electrical Engineering, Yogyakarta, Indonesia, 2020: 244-248.
4	Satti A T, Kim J, Yi E, et al. Microneedle array electrode-based wearable EMG system for detection of driver drowsiness through steering wheel grip[J]. Sensors, 2021, 21(15): 5091-5104.
5	Ebrahimian S, Nahvi A, Tashakori M, et al. Multi-level classification of driver drowsiness by simultaneous analysis of ECG and respiration signals using deep neural networks[J]. International Journal of Environmental Research and Public Health, 2022, 19(17): 10736-10752.
6	Jantan S, Ahmad S A, Soh A C, et al. A multi-model analysis for driving fatigue detection using EEG signals[C]∥Proceedings of the 7th IEEE-EMBS Conference on Biomedical Engineering and Sciences(IECBES), Kuala Lumpur, Malaysia, 2022: 183-188.
7	Pakdamanian E, Sheng S, Baee S, et al. DeepTake: prediction of driver takeover behavior using multimodal data[C]∥Proceedings of the CHI Conference on Human Factors in Computing Systems, Yokohama,Japan, 2021:No.3445563.
8	Wang X, Xu R, Zhang S, et al. Driver distraction detection based on vehicle dynamics using naturalistic driving data[J]. Transportation Research Part C: Emerging Technologies, 2022, 136(2022): 103561-103572.
9	Li Z, Chen L, Nie L, et al. A novel learning model of driver fatigue features representation for steering wheel angle[J]. IEEE Transactions on Vehicular Technology, 2022, 71(1): 269-281.
10	Tang M, Wu F, Zhao L L, et al. Detection of distracted driving based on MultiGranularity and Middle-Level features[C]∥Chinese Automation Congress (CAC),Shanghai, China, 2020: 2717-2722.
11	Lin Y, Cao D, Fu Z, et al. A lightweight attention-based network towards distracted driving behavior recognition[J]. Applied Sciences, 2022, 12(9): 4191-4208.
12	Huang T, Fu R, Chen Y, et al. Real-time driver behavior detection based on deep deformable inverted residual network with an attention mechanism for human-vehicle co-driving system[J]. IEEE Transactions on Vehicular Technology, 2022, 71(12): 12475-12488.
13	Liu D, Yamasaki T, Wang Y, et al. Toward extremely lightweight distracted driver recognition with distillation-based neural architecture search and knowledge transfer[J]. IEEE Transactions on Intelligent Transportation Systems, 2022, 24(1): 764-777.
14	He K, Zhang X, Ren S, et al. Deep residual learning for image recognition[C]∥Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, USA,2016:770-778.
15	Deng J, Dong W, Socher R, et al. Imagenet: a large-scale hierarchical image database[C]∥Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Miami, USA,2009:248-255.
16	Dai J F, Qi H Z, Xiong Y W, et al. Deformable convolutional networks[C]∥Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy,2017:764-773.
17	Szegedy C, Liu W, Jia Y, et al. Going deeper with convolutions[C]∥Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, USA, 2015:No.7298594.
18	Inc Kaggle. State farm distracted driver detection[DB/OL].[2023-03-25]..
19	Omeiza D, Speakman S, Cintas C, et al. Smooth grad-cam++: an enhanced inference level visualization technique for deep convolutional neural network models [J/OL].[2023-03-26]. .
20	Krizhevsky A, Sutskever I, Hinton G E. Imagenet classification with deep convolutional neural networks [J]. Communications of the ACM, 2017, 60(6): 84-90.
21	Huang G, Liu Z, Van Der Maaten L, et al. Densely connected convolutional networks[C]∥Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu,USA,2017: 4700-4708.
22	Szegedy C, Vanhoucke V, Ioffe S, et al. Rethinking the inception architecture for computer vision[C]∥Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas,USA,2016: 2818-2826.
23	Su J, Vargas D V, Sakurai K. One pixel attack for fooling deep neural networks[J]. IEEE Transactions on Evolutionary Computation, 2019, 23(5): 828-841.

相关文章 15

[1]	王军年,曹宇靖,罗智仁,李凯旋,赵文伯,孟盈邑. 基于双目视觉的道路水深在线检测算法[J]. 吉林大学学报(工学版), 2025, 55(1): 175-184.
[2]	汪豪,赵彬,刘国华. 基于时间和运动增强的视频动作识别[J]. 吉林大学学报(工学版), 2025, 55(1): 339-346.
[3]	谭草,任浩鑫,葛文庆,宋亚东,陆佳瑜. 直驱阀控液压振动平台改进自抗扰控制[J]. 吉林大学学报(工学版), 2025, 55(1): 84-92.
[4]	李寿涛,杨路,屈如意,孙鹏鹏,于丁力. 基于模型预测控制的滑移率控制方法[J]. 吉林大学学报(工学版), 2024, 54(9): 2687-2696.
[5]	吴量,顾义凡,邢彪,马芳武,倪利伟,贾微微. 基于线性二次型调节器的四轮转向与分布式集成控制方法[J]. 吉林大学学报(工学版), 2024, 54(9): 2414-2422.
[6]	特木尔朝鲁朝鲁,张亚萍. 基于卷积神经网络的无线传感器网络链路异常检测算法[J]. 吉林大学学报(工学版), 2024, 54(8): 2295-2300.
[7]	朱圣杰,王宣,徐芳,彭佳琦,王远超. 机载广域遥感图像的尺度归一化目标检测方法[J]. 吉林大学学报(工学版), 2024, 54(8): 2329-2337.
[8]	王玉海,李晓之,李兴坤. 面向高速工况的混合动力卡车预见性节能算法[J]. 吉林大学学报(工学版), 2024, 54(8): 2121-2129.
[9]	常胜,刘宏飞,邹乃威. 汽车变曲率路径循迹H_∞回路成形鲁棒控制[J]. 吉林大学学报(工学版), 2024, 54(8): 2141-2148.
[10]	赵宏伟,武鸿,马克,李海. 基于知识蒸馏的图像分类框架[J]. 吉林大学学报(工学版), 2024, 54(8): 2307-2312.
[11]	张锦洲,姬世青,谭创. 融合卷积神经网络和双边滤波的相贯线焊缝提取算法[J]. 吉林大学学报(工学版), 2024, 54(8): 2313-2318.
[12]	刘建泽,柳江,李敏,章新杰. 基于最小二乘的车速解耦路面辨识方法[J]. 吉林大学学报(工学版), 2024, 54(7): 1821-1830.
[13]	谢宪毅,张明君,金立生,周彬,胡涛,白宇飞. 考虑舒适度的智能汽车人工蜂群轨迹规划方法[J]. 吉林大学学报(工学版), 2024, 54(6): 1570-1581.
[14]	孙铭会,薛浩,金玉波,曲卫东,秦贵和. 联合时空注意力的视频显著性预测[J]. 吉林大学学报(工学版), 2024, 54(6): 1767-1776.
[15]	刘从臻,陈高,刘洪柱,马强,徐成伟,孟辉,王国林. 湿滑路面轮胎接地力学特性模型[J]. 吉林大学学报(工学版), 2024, 54(6): 1501-1511.

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed

网络	C0	C1	C2	C3	C4	C5	C6	C7	C8	C9	准确率
AlexNet	85.93	98.31	97.25	98.39	86.78	87.27	84.12	99.66	60.35	64.34	85.93
DenseNet	80.78	98.67	100.0	99.57	97.50	98.73	98.94	99.70	85.40	84.53	94.67
InceptionV3	78.89	99.10	100.0	99.35	99.57	99.58	97.93	99.13	89.52	83.53	94.86
VGG16	57.29	94.15	94.57	96.46	90.06	69.87	94.92	100.0	53.17	71.43	80.02
本文	84.66	99.55	99.79	99.15	98.52	100.0	99.37	99.42	92.00	88.16	96.23

网络模型	test 1	test 2	test 3	test 4	test 5	test 6	test 7	test 8	test 9	test 10	平均值
AlexNet	201.1	182.9	182.6	197.6	183.9	196.3	192.2	193.6	193.9	191.6	191.6
DenseNet	43.23	40.29	41.62	42.18	43.92	43.2	42.26	44.43	42.48	43.74	42.74
Inception V3	54.64	53.95	55.62	54.31	54.2	55.22	54.9	54.16	55.86	55.97	54.88
VGG16	53.56	52.69	52.71	51.72	52.98	54.29	53.2	52.72	53.65	52.66	53.02
本文	59.11	59.04	58.57	58.9	57.98	57.61	57.61	58.53	59.04	58.6	58.5