多模态数据在驾驶疲劳监测中的有效性分析

doi:10.13229/j.cnki.jdxbgxb.20240877

Abstract

Abstract:

The existing research on multimodal driving fatigue detection mainly focuses on the application level， lacking in-depth exploration of the underlying mechanisms between each mode and driving fatigue. This study aims to explore the relationship between commonly used data sources （including electroencephalogram， electrocardiogram， and vehicle motion information parameters） and driving fatigue， and analyze the potential correlation between these data sources in driving fatigue detection. We collected 32 sets of data through driving simulation experiments and evaluated subjective fatigue levels using the Karolinska Sleepiness Scale， with eyelid closure as an indicator of objective fatigue levels. The parameters of the three modalities are the standard deviation of the R-peak interval of the electrocardiogram （RMSSD）， the power ratio of the EEG frequency band （α+θ/β）， and the standard deviation of the vehicle lateral offset （SDLP）. In the exploratory factor analysis results， the variance explained by the first three factors exceeds 50%. A potential relationship model between various modes and driving fatigue was constructed using structural equation modeling. The analysis results showed that each mode can explain the variability of driving fatigue to a certain extent. Among them， RMSSD and SDLP have significant advantages in predicting subjective fatigue， while α+θ/β shows a close correlation with objective fatigue.

Key words: engineering of communication and transportation system, driving fatigue, multimodal, structural equation model

CLC Number:

U491

Yu SUN,Shi-wu LI,Meng-zhu GUO,Tong-tong JIN,Hui-jun SONG,De-zhi LIU,Wen GAO. Analysis of effectiveness of multimodal data in driving fatigue detection[J].Journal of Jilin University(Engineering and Technology Edition), 2026, 56(2): 473-479.

Figures/Tables 11

Fig.1

Fig.2

Table 1

Table 2

Table 3

Table 4

Table 5

Table 6

Fig.3

Fig.4

Fig.5

References 14

[1]	苏瑞芝, 唐巾卜, 阿地力·吐合提, 等. 基于生理参数的驾驶疲劳检测方法综述[J]. 复旦学报: 自然科学版, 2023, 62(4): 419-427.
	Su Rui-zhi, Tang Jin-bu, Tuheti Adili, et al. Review of driving fatigue detection methods based on physiological parameters[J]. Fudan Journal (Natural Science Edition), 2023, 62(4): 419-427.
[2]	金立生, 牛清宁, 刘景华, 等. 不同道路线形下驾驶人认知分散状态监测[J]. 吉林大学学报: 工学版, 2014, 44(3): 642-647.
	Jin Li-sheng, Niu Qing-ning, Liu Jing-hua, et al. Driver cognitive distraction detection in different road lines[J]. Journal of Jilin University(Engineering and Technology Edition),2014,44(3): 642-647.
[3]	孙一帆, 吴超仲, 张晖, 等. 个体差异对转向指标疲劳辨识能力的影响分析[J]. 中国公路学报, 2020, 33(6): 157-167.
	Sun Yi-fan, Wu Chao-zhong, Zhang Hui, et al. Analysis of the influence of individual differences on the fatigue identification ability of steering indicators[J]. Chinese Journal of Highways, 2020, 33 (6): 157-167.
[4]	Wang Y, Liu X, Zhang Y, et al. Driving fatigue detection based on EEG signal[C]∥Fifth International Conference on Instrumentation and Measurement, Computer, Communication and Control, Qinhuangdao, China, 2015: 715-718.
[5]	Lan Z, Zhao J, Liu P, et al. Driving fatigue detection based on fusion of EEG and vehicle motion information[J]. Biomedical Signal Processing and Control, 2024, 92: 106031.
[6]	Wang H, Dragomir A, Abbasi N I, et al. A novel real-time driving fatigue detection system based on wireless dry EEG[J]. Cognitive Neurodynamics, 2018, 12(4): 365-376.
[7]	Chen J, Wang H, Wang Q, et al. Exploring the fatigue affecting electroencephalography based functional brain networks during real driving in young males [J]. Neuropsychologia, 2019, 129: 200-211.
[8]	Harvy J, Bezerianos A, Li J. Reliability of EEG measures in driving fatigue[J]. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 2022, 30: 2743-2753.
[9]	Van R C M, Kollee L A, Hopman J C, et al. Heart rate variability[J]. Annals of Internal Medicine, 1993, 118(6): 436-447.
[10]	Bhardwaj R, Natrajan P, Balasubramanian V. Study to determine the effectiveness of deep learning classifiers for ECG based driver fatigue classification[C]∥Proceedings of the IEEE 13th International Conference on Industrial and Information Systems, Rupnagar, India, 2018: 98-102..
[11]	Zhang X, Wang X, Yang X, et al. Driver drowsiness detection using mixed-effect ordered logit model considering time cumulative effect[J]. Analytic Methods in Accident Research, 2020, 26: 100114.
[12]	Hu X, Lodewijks G. Exploration of the effects of task-related fatigue on eye-motion features and its value in improving driver fatigue-related technology[J]. Transportation Research Part F: Traffic Psychology and Behaviour, 2021, 80: 150-171.
[13]	Zhang H, Ni D, Ding N, et al. Structural analysis of driver fatigue behavior: a systematic review[J]. Transportation Research Interdisciplinary Perspectives, 2023, 21: 100865.
[14]	Srinivasan A G, Smith S S, Pattinson C L, et al. Heart rate variability as an indicator of fatigue: a structural equation model approach[J]. Transportation Research Part F: Traffic Psychology and Behaviour, 2024, 103: 420-429.

Related Articles 15

[1]	Wen-hui ZHANG,Mei-ru YE,Cong XI,Zi-wen SONG. Vehicle formation and carbon emission characteristics in mixed traffic flow environment [J]. Journal of Jilin University(Engineering and Technology Edition), 2026, 56(2): 416-430.
[2]	Yi-yong PAN,Tian-yu CAO,Yu LIU. Constrained most reliable path in stochastic traffic network alternating direction method of multipliers [J]. Journal of Jilin University(Engineering and Technology Edition), 2026, 56(2): 455-463.
[3]	Yu-sheng CI,Yi-kang HUANG. Overview of intersection vehicle-infrastructure integration based on bibliometrics [J]. Journal of Jilin University(Engineering and Technology Edition), 2026, 56(2): 313-332.
[4]	De-xin YU,Lu-chen WANG,Xin-cheng WU,Jian-yu MAO,Shi-long SHI. Multi-layer time dependent network path planning algorithm considering waiting time [J]. Journal of Jilin University(Engineering and Technology Edition), 2026, 56(2): 443-454.
[5]	Zhuang-lin MA,Yu-ming BI,Bei ZHOU,Ya-juan DENG,Xue ZHAO. Heterogeneity analysis of residents’ transfer intentions under transit transfer preferential policy [J]. Journal of Jilin University(Engineering and Technology Edition), 2026, 56(1): 158-169.
[6]	Zhao-wei QU,Ming-yang WANG,Zhe WANG,Xian-min SONG,Yun-xiang ZHANG,Jing-cheng HUANG. Adaptive scheduling method for bus based on autonomous modular vehicles main⁃auxiliary function allocation [J]. Journal of Jilin University(Engineering and Technology Edition), 2025, 55(9): 2946-2957.
[7]	Chang-ru MU,Liang XU,Guo-zhu CHENG. Collision prevention performance of outsourced U-shaped steel concrete composite guardrail based on reasonable energy allocation [J]. Journal of Jilin University(Engineering and Technology Edition), 2025, 55(8): 2669-2680.
[8]	Yuan-ning LIU,Xing-zhe WANG,Zi-yu HUANG,Jia-chen ZHANG,Zhen LIU. Stomach cancer survival prediction model based on multimodal data fusion [J]. Journal of Jilin University(Engineering and Technology Edition), 2025, 55(8): 2693-2702.
[9]	Yan-bo LI,Jing-yuan WANG,Yuan-yuan Chen,Shao-feng CHENG,Hao-nan LYU,Jun-shuo CHEN. RAMS assessment approach of self-consistent energy system in highway service areas [J]. Journal of Jilin University(Engineering and Technology Edition), 2025, 55(7): 2243-2250.
[10]	Xiao-feng JI,Ruo-fan DENG,Xin QIAO,Hao-tian GUAN. Nonlinear influence of built environment on temporal aggregation modes of shared bicycles [J]. Journal of Jilin University(Engineering and Technology Edition), 2025, 55(7): 2233-2242.
[11]	Sheng-yu YAN,Ming-jie CHENG,Hong-ce TIAN,Hong-yu WANG,Yong-heng ZHOU,Bo-hao MA. Scheduling algorithm for battery electric vehicle in closed scenic area [J]. Journal of Jilin University(Engineering and Technology Edition), 2025, 55(6): 1984-1993.
[12]	Yan-yan QIN,Teng-fei XIAO,Qin-zhong LUO,Bao-jie WANG. Car-following safety analysis and control strategy for foggy freeway [J]. Journal of Jilin University(Engineering and Technology Edition), 2025, 55(4): 1241-1249.
[13]	Yi-yong PAN,Xiang-yu XU. Model for predicting severity of accidents based on MobileViT network considering imbalanced data [J]. Journal of Jilin University(Engineering and Technology Edition), 2025, 55(3): 947-953.
[14]	Yong-zheng YANG,Zhi-gang DU,Jia-lin MEI. Setting method and effect evaluation of linear guiding system in highway tunnels [J]. Journal of Jilin University(Engineering and Technology Edition), 2025, 55(12): 3885-3897.
[15]	Guang-yong CHEN,Shi-rui ZHOU,Chu-qing TAO,Li WAN,Wei WEI. Collaborative control of tunnel speed and lighting based on driver’s visual characteristics [J]. Journal of Jilin University(Engineering and Technology Edition), 2025, 55(12): 3898-3906.

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed

Comments

Recommended 0

No Suggested Reading articles found!

数据来源	参数
驾驶员主观疲劳	KSS得分
驾驶员客观疲劳	PERCLOS
脑电图	α+θ/β
心电图	RMSSD
车辆运动信息	SDLP
屏幕警惕任务	反应时间、任务完成准确率

	场景1：短时间重负载				场景2：长时间轻负载
	1 h含次要任务驾驶				1 h单调驾驶
	驾驶前		驾驶后		驾驶前		驾驶后
	M	SD	M	SD	M	SD	M	SD
KSS	3.30	1.55	5.45	1.78	3.15	1.65	6.35	1.23
WP	-	-	6.55	1.52	-	-	2.45	1.44

	场景1：短时间重负载				场景2：长时间轻负载
	1 h含次要任务驾驶				1 h单调驾驶
	驾驶前		驾驶后		驾驶前		驾驶后
	M	SD	M	SD	M	SD	M	SD
WM	3.88	0.89	4.43	1.11	3.91	0.98	3.96	0.92
CI	1.33	0.43	1.44	0.39	1.27	0.41	1.47	0.41
RC	0.33	0.05	0.40	0.07	0.32	0.06	0.38	0.11
A	6.91	1.49	9.93	2.86	7.08	1.75	9.15	2.51

信号	判断有	判断无
有	击中	Ⅰ类错误
无	Ⅱ类错误	正确拒绝

	场景1：短时间重负载				场景2：长时间轻负载
	1 h含次要任务驾驶				1 h单调驾驶
	驾驶前		驾驶后		驾驶前		驾驶后
	M	SD	M	SD	M	SD	M	SD
WM	0.75	0.02	0.68	0.02	0.79	0.02	0.77	0.02
CI	0.96	0.01	0.99	0.01	0.96	0.01	0.96	0.01
RC	-	-	-	-	-	-	-	-
A	0.92	0.02	0.72	0.02	0.9.	0.02	0.79	0.01

Analysis of effectiveness of multimodal data in driving fatigue detection

RICH HTML

PDF (PC)