多模态数据在驾驶疲劳监测中的有效性分析

doi:10.13229/j.cnki.jdxbgxb.20240877

摘要/Abstract

摘要：

现有的多模态驾驶疲劳研究主要集中于应用层面，缺乏对各模态与驾驶疲劳之间内在机理的深入探讨。本文旨在探究常用数据源（包括脑电图、心电图及车辆运动信息参数）与驾驶疲劳之间的关系，分析这些数据源在驾驶疲劳中的潜在关联性。通过驾驶模拟试验收集了32组数据，并使用卡罗琳斯卡嗜睡量表评估主观疲劳等级，以眼睑闭合度作为客观疲劳等级的指标。3个模态中参与分析的参数分别为心电R峰间期的标准差（RMSSD）、脑电频段功率比（α+θ/β）以及车辆横向偏移的标准差（SDLP）。探索性因子分析结果中，前3个因子的方差解释率超过50%。采用结构方程模型构建了各模态与驾驶疲劳之间的潜在关系模型，分析结果表明：各模态均能在一定程度上解释驾驶疲劳的变异性，其中，RMSSD和SDLP在预测主观疲劳方面具有显著优势，而α+θ/β与客观疲劳表现出密切的关联性。

关键词: 交通运输系统工程, 驾驶疲劳, 多模态, 结构方程模型

Abstract:

The existing research on multimodal driving fatigue detection mainly focuses on the application level， lacking in-depth exploration of the underlying mechanisms between each mode and driving fatigue. This study aims to explore the relationship between commonly used data sources （including electroencephalogram， electrocardiogram， and vehicle motion information parameters） and driving fatigue， and analyze the potential correlation between these data sources in driving fatigue detection. We collected 32 sets of data through driving simulation experiments and evaluated subjective fatigue levels using the Karolinska Sleepiness Scale， with eyelid closure as an indicator of objective fatigue levels. The parameters of the three modalities are the standard deviation of the R-peak interval of the electrocardiogram （RMSSD）， the power ratio of the EEG frequency band （α+θ/β）， and the standard deviation of the vehicle lateral offset （SDLP）. In the exploratory factor analysis results， the variance explained by the first three factors exceeds 50%. A potential relationship model between various modes and driving fatigue was constructed using structural equation modeling. The analysis results showed that each mode can explain the variability of driving fatigue to a certain extent. Among them， RMSSD and SDLP have significant advantages in predicting subjective fatigue， while α+θ/β shows a close correlation with objective fatigue.

Key words: engineering of communication and transportation system, driving fatigue, multimodal, structural equation model

中图分类号:

U491

孙宇,李世武,郭梦竹,金桐彤,宋会军,刘德志,高雯. 多模态数据在驾驶疲劳监测中的有效性分析[J]. 吉林大学学报(工学版), 2026, 56(2): 473-479.

Yu SUN,Shi-wu LI,Meng-zhu GUO,Tong-tong JIN,Hui-jun SONG,De-zhi LIU,Wen GAO. Analysis of effectiveness of multimodal data in driving fatigue detection[J]. Journal of Jilin University(Engineering and Technology Edition), 2026, 56(2): 473-479.

图/表 11

图1

图2

表1

表2

表3

表4

表5

表6

图3

图4

图5

参考文献 14

[1]	苏瑞芝, 唐巾卜, 阿地力·吐合提, 等. 基于生理参数的驾驶疲劳检测方法综述[J]. 复旦学报: 自然科学版, 2023, 62(4): 419-427.
	Su Rui-zhi, Tang Jin-bu, Tuheti Adili, et al. Review of driving fatigue detection methods based on physiological parameters[J]. Fudan Journal (Natural Science Edition), 2023, 62(4): 419-427.
[2]	金立生, 牛清宁, 刘景华, 等. 不同道路线形下驾驶人认知分散状态监测[J]. 吉林大学学报: 工学版, 2014, 44(3): 642-647.
	Jin Li-sheng, Niu Qing-ning, Liu Jing-hua, et al. Driver cognitive distraction detection in different road lines[J]. Journal of Jilin University(Engineering and Technology Edition),2014,44(3): 642-647.
[3]	孙一帆, 吴超仲, 张晖, 等. 个体差异对转向指标疲劳辨识能力的影响分析[J]. 中国公路学报, 2020, 33(6): 157-167.
	Sun Yi-fan, Wu Chao-zhong, Zhang Hui, et al. Analysis of the influence of individual differences on the fatigue identification ability of steering indicators[J]. Chinese Journal of Highways, 2020, 33 (6): 157-167.
[4]	Wang Y, Liu X, Zhang Y, et al. Driving fatigue detection based on EEG signal[C]∥Fifth International Conference on Instrumentation and Measurement, Computer, Communication and Control, Qinhuangdao, China, 2015: 715-718.
[5]	Lan Z, Zhao J, Liu P, et al. Driving fatigue detection based on fusion of EEG and vehicle motion information[J]. Biomedical Signal Processing and Control, 2024, 92: 106031.
[6]	Wang H, Dragomir A, Abbasi N I, et al. A novel real-time driving fatigue detection system based on wireless dry EEG[J]. Cognitive Neurodynamics, 2018, 12(4): 365-376.
[7]	Chen J, Wang H, Wang Q, et al. Exploring the fatigue affecting electroencephalography based functional brain networks during real driving in young males [J]. Neuropsychologia, 2019, 129: 200-211.
[8]	Harvy J, Bezerianos A, Li J. Reliability of EEG measures in driving fatigue[J]. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 2022, 30: 2743-2753.
[9]	Van R C M, Kollee L A, Hopman J C, et al. Heart rate variability[J]. Annals of Internal Medicine, 1993, 118(6): 436-447.
[10]	Bhardwaj R, Natrajan P, Balasubramanian V. Study to determine the effectiveness of deep learning classifiers for ECG based driver fatigue classification[C]∥Proceedings of the IEEE 13th International Conference on Industrial and Information Systems, Rupnagar, India, 2018: 98-102..
[11]	Zhang X, Wang X, Yang X, et al. Driver drowsiness detection using mixed-effect ordered logit model considering time cumulative effect[J]. Analytic Methods in Accident Research, 2020, 26: 100114.
[12]	Hu X, Lodewijks G. Exploration of the effects of task-related fatigue on eye-motion features and its value in improving driver fatigue-related technology[J]. Transportation Research Part F: Traffic Psychology and Behaviour, 2021, 80: 150-171.
[13]	Zhang H, Ni D, Ding N, et al. Structural analysis of driver fatigue behavior: a systematic review[J]. Transportation Research Interdisciplinary Perspectives, 2023, 21: 100865.
[14]	Srinivasan A G, Smith S S, Pattinson C L, et al. Heart rate variability as an indicator of fatigue: a structural equation model approach[J]. Transportation Research Part F: Traffic Psychology and Behaviour, 2024, 103: 420-429.

相关文章 15

[1]	张文会,叶梅茹,席聪,宋子文. 混合交通流环境下车辆编队与碳排放特性[J]. 吉林大学学报(工学版), 2026, 56(2): 416-430.
[2]	潘义勇,曹天宇,刘宇. 随机交通网络约束最可靠路径乘子交替方向法[J]. 吉林大学学报(工学版), 2026, 56(2): 455-463.
[3]	慈玉生,黄轶康. 基于文献计量的交叉口车路协同研究综述[J]. 吉林大学学报(工学版), 2026, 56(2): 313-332.
[4]	马壮林,毕宇明,周备,邓亚娟,兆雪. 公交换乘优惠政策下居民换乘意向的异质性分析[J]. 吉林大学学报(工学版), 2026, 56(1): 158-169.
[5]	郭艳萍,高云,周建慧. 基于链路状态的卫星通信多模态动态拥塞控制算法[J]. 吉林大学学报(工学版), 2026, 56(1): 257-264.
[6]	曲昭伟,王铭阳,王喆,宋现敏,张云翔,黄镜尘. 基于自动驾驶模块化车辆主辅功能分配的公交自适应调度方法[J]. 吉林大学学报(工学版), 2025, 55(9): 2946-2957.
[7]	王琳虹,刘宇阳,刘子昱,鹿应佳,张宇恒,黄桂树. 基于YOLOv5的轻量化桥梁缺陷识别[J]. 吉林大学学报(工学版), 2025, 55(9): 2958-2968.
[8]	张云翔,宋现敏,谢渝,湛天舒. 基于用户满意度的停车预约服务智能体行为仿真[J]. 吉林大学学报(工学版), 2025, 55(9): 2978-2984.
[9]	穆长儒,徐亮,程国柱. 基于能量合理分配的外包U型钢-混凝土组合护栏防撞性能[J]. 吉林大学学报(工学版), 2025, 55(8): 2669-2680.
[10]	刘元宁,王星喆,黄子彧,张家晨,刘震. 基于多模态数据融合的胃癌患者生存预测模型[J]. 吉林大学学报(工学版), 2025, 55(8): 2693-2702.
[11]	李艳波,汪静远,陈圆媛,程绍峰,吕浩楠,陈俊硕. 面向高速公路服务区自洽能源系统的RAMS评价方法[J]. 吉林大学学报(工学版), 2025, 55(7): 2243-2250.
[12]	柴树山,周志强,李海涛,徐炅旸. 基于图时空模式学习网络的路网实时交通事件自动检测方法[J]. 吉林大学学报(工学版), 2025, 55(7): 2145-2161.
[13]	于江波,翁剑成,林鹏飞,孙宇星,柴娇龙. 基于混合Transformer的对外客运枢纽抵站客流预测模型[J]. 吉林大学学报(工学版), 2025, 55(7): 2251-2259.
[14]	戢晓峰,邓若凡,乔新,关昊天. 建成环境对共享单车时间集聚模式的非线性影响[J]. 吉林大学学报(工学版), 2025, 55(7): 2233-2242.
[15]	闫晟煜,程铭杰,田宏策,王洪瑀,周永恒,马博浩. 封闭式景区纯电动客车调度方法[J]. 吉林大学学报(工学版), 2025, 55(6): 1984-1993.

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed

数据来源	参数
驾驶员主观疲劳	KSS得分
驾驶员客观疲劳	PERCLOS
脑电图	α+θ/β
心电图	RMSSD
车辆运动信息	SDLP
屏幕警惕任务	反应时间、任务完成准确率

	场景1：短时间重负载				场景2：长时间轻负载
	1 h含次要任务驾驶				1 h单调驾驶
	驾驶前		驾驶后		驾驶前		驾驶后
	M	SD	M	SD	M	SD	M	SD
KSS	3.30	1.55	5.45	1.78	3.15	1.65	6.35	1.23
WP	-	-	6.55	1.52	-	-	2.45	1.44

	场景1：短时间重负载				场景2：长时间轻负载
	1 h含次要任务驾驶				1 h单调驾驶
	驾驶前		驾驶后		驾驶前		驾驶后
	M	SD	M	SD	M	SD	M	SD
WM	3.88	0.89	4.43	1.11	3.91	0.98	3.96	0.92
CI	1.33	0.43	1.44	0.39	1.27	0.41	1.47	0.41
RC	0.33	0.05	0.40	0.07	0.32	0.06	0.38	0.11
A	6.91	1.49	9.93	2.86	7.08	1.75	9.15	2.51

信号	判断有	判断无
有	击中	Ⅰ类错误
无	Ⅱ类错误	正确拒绝

	场景1：短时间重负载				场景2：长时间轻负载
	1 h含次要任务驾驶				1 h单调驾驶
	驾驶前		驾驶后		驾驶前		驾驶后
	M	SD	M	SD	M	SD	M	SD
WM	0.75	0.02	0.68	0.02	0.79	0.02	0.77	0.02
CI	0.96	0.01	0.99	0.01	0.96	0.01	0.96	0.01
RC	-	-	-	-	-	-	-	-
A	0.92	0.02	0.72	0.02	0.9.	0.02	0.79	0.01