Journal of Jilin University(Engineering and Technology Edition) ›› 2023, Vol. 53 ›› Issue (3): 758-771.doi: 10.13229/j.cnki.jdxbgxb20210919

Previous Articles    

Drivers' takeover behavior and intention recognition based on factor and long short⁃term memory

Rong-han YAO1(),Wen-tao XU1,Wei-wei GUO2   

  1. 1.School of Transportation and Logistics，Dalian University of Technology，Dalian 116024，China
    2.Beijing Key Laboratory of Urban Intelligent Traffic Control Technology，North China University of Technology，Beijing 100144，China
  • Received:2021-09-14 Online:2023-03-01 Published:2023-03-29

RICH HTML

 9   

PDF (PC)

408

Abstract:

To identify drivers’ takeover behavior and intention in an autonomous driving environment， with the help of a driving simulator and an eye-tracking device， the simulation tests were conducted to let drivers takeover an autonomous vehicle ten times in five emergency situations on a two-way six-lane freeway of 18.95 km. Using the vehicle operation and visual attention data， the three common factors were extracted by the factor analysis， and the K-means clustering analysis was used to qualitatively identify the drivers' takeover behavior and intention. The factor analysis was respectively combined with the support vector machine and the long short-term memory neural network， then the two models were obtained to quantitatively identify drivers' takeover behavior and intention. Research results show that： drivers' takeover behavior is influenced by their longitudinal response， lateral response and visual attention； the clustering analysis can qualitatively describe the takeover behavior and intention of different types of drivers and reveal the potential driving safety risks； compared with the support vector machine， the long short-term memory neural network and the factor and support vector machine model， the factor and long short-term memory model is more effective in identifying drivers' takeover intention， with the best four performance indices of accuracy rate， recall rate， F1-score and precision rate； and the common factors which are obtained using the factor analysis for data downscaling and effective information enrichment are helpful to improve the classification performance of the driving takeover intention recognition model. This study is helpful to identify drivers who are at a higher risk of takeover and to design some targeted driving assistance strategies.

Key words: engineering of communications and transportation system, drivers' takeover behavior and intention, factor analysis, K-means clustering analysis, long short-term memory neural network

CLC Number: 

  • U491.2

Fig.1

Equipment used for takeover tests"

Fig.2

Emergent locations and corresponding emergency situations for takeover incidents"

Table 1

Experimental results of participants taking over an autonomous vehicle"

参与者模拟试验正式试验
1接管质量好出现事件6和10时，自动驾驶车辆与前方车辆碰撞
2没及时接管出现事件6和10时，自动驾驶车辆转角过大
3接管质量好接管质量好
4接管质量好出现事件3和6时，自动驾驶车辆分别与故障车辆和前方车辆发生碰撞
6接管质量好出现事件3时，自动驾驶车辆转角过大
7接管质量好接管质量好
8车辆转角过小接管质量好
9接管质量好出现事件1和3时，自动驾驶车辆转角过大
10接管质量好出现事件3时，自动驾驶车辆转角过大；出现事件8时，自动驾驶车辆遇限速标志而没减速
11接管质量差出现事件2时，自动驾驶车辆转角过大；出现事件3时，自动驾驶车辆与故障车辆发生碰撞；出现事件4、5和6时，自动驾驶车辆刹停；出现事件8时，自动驾驶车辆遇限速标志而没减速

Fig.3

Variations of takeover behavior parameters with driving distance when participant 7 dealing with first takeover event"

Table 2

Dependent variables selected fortakeover behavior analysis"

因变量定义
视觉注意力数据左眼瞳孔直径标准差/mm左眼瞳孔直径的离散程度
右眼瞳孔直径标准差/mm右眼瞳孔直径的离散程度
凝视行为/%视线落在道路上的时间百分比
车辆运行数据速度标准差/（km?h-1）车辆速度的离散程度
最大速度差/（km?h-1）车辆速度的变化幅度
平均速度/（km?h-1）车辆速度的平均快慢程度
最大纵向加速度/（m?s-2）接管过程中车辆最大制动加速度
最大横摆角/rad车辆横摆角的变化幅度
横摆角标准差/rad车辆横摆角的离散程度

Table 3

Bartlett's spherical test and KMO test"

检验指标结果
巴特利特球形检验近似卡方875.790
自由度36
显著性0.000
KMO取样适切性量数0.672

Table 4

Total variance explained by factor analysis"

成分特征值方差百分比/%累积方差百分比/%
13.69841.08641.086
22.39526.60867.694
31.30614.51282.206
40.6577.29689.502
50.4424.91394.415
60.3043.37797.792
70.1021.13398.925
80.0870.96299.887
90.0100.113100.000

Table 5

Component matrix obtained by factor analysis"

原始指标因子载荷值
公因子1公因子2公因子3
速度标准差/（km?h-1）0.9670.0430.155
最大速度差/（km?h-1）0.9550.0970.200
平均速度/（km?h-1）-0.8900.048-0.082
最大纵向加速度/（m?s-2）0.7920.0080.277
左眼瞳孔直径标准差/mm0.2250.793-0.288
最大横摆角/rad-0.3210.7470.520
右眼瞳孔直径标准差/mm0.2870.719-0.363
横摆角标准差/rad-0.4360.6470.576
凝视行为/%-0.077-0.5080.584

Table 6

Rotated component matrix"

原始指标因子载荷值
公因子1公因子2公因子3
最大速度差/（km?h-1）0.9710.122-0.059
速度标准差/（km?h-1）0.9650.112-0.128
平均速度/（km?h-1）-0.864-0.0760.218
最大纵向加速度/（m?s-2）0.838-0.023-0.018
右眼瞳孔直径标准差/mm0.1840.8230.138
左眼瞳孔直径标准差/mm0.1540.8210.257
凝视行为/%0.096-0.7660.098
横摆角标准差/rad-0.1940.0430.950
最大横摆角/rad-0.1000.1750.944

Table 7

Factor score coefficient matrix"

原始指标因子得分系数
公因子1公因子2公因子3
左眼瞳孔直径标准差/mm0.0000.4000.046
右眼瞳孔直径标准差/mm-0.0040.416-0.019
凝视行为/%0.117-0.4460.181
平均速度/（km?h-1）-0.2470.0100.035
速度标准差/（km?h-1）0.286-0.0130.024
最大速度差/（km?h-1）0.296-0.0190.063
最大纵向加速度/（m?s-2）0.272-0.0920.092
横摆角标准差/rad0.045-0.0970.520
最大横摆角/rad0.062-0.0330.509

Fig.4

K-means clustering results"

Table 8

Clustering results and profile values when using drivers' takeover longitudinal response as a clustering indicator"

类别样本量试验编号接管强度轮廓值
平均值最小值最大值
1217，8，16，46，51，53，56，57，58，59，60，75，81，85，86，88，93，94，95，96，98强0.5800.1230.723
2371，4，9，10，17，20，26，27，28，29，31，34，38，45，47，49，50，52，54，55，61，64，67，69，71，72，73，74，76，77，78，79，83，84，89，97，99一般0.5070.0650.735
3412，3，5，6，11，12，13，14，15，18，19，21，22，23，24，25，30，32，33，35，36，37，39，40，41，42，43，44，48，62，63，65，66，68，70，80，82，87，90，91，92弱0.7110.0620.829

Table 9

Clustering results and profile values when using drivers' takeover lateral response as a clustering indicator"

类别样本量试验编号接管强度轮廓值
平均值最小值最大值
1202，3，5，6，16，17，20，33，42，43，47，52，53，55，72，75，81，82，88，99强0.4820.0820.620
2681，4，7，8，9，10，11，12，13，14，15，19，21，22，23，24，25，26，27，29，30，31，32，34，35，36，37，40，41，44，45，46，49，51，54，56，57，59，60，61，62，63，64，65，66，68，69，70，71，73，74，76，78，79，80，83，84，85，86，89，90，91，92，93，94，95，96，98一般0.5960.0250.773
31118，28，38，39，48，50，58，67，77，87，97弱0.6210.0450.789

Table 10

Clustering results and profile values when using drivers' takeover visual attention as a clustering indicator"

类别样本量试验编号接管强度轮廓值
平均值最小值最大值
1333，5，8，9，13，14，15，19，25，29，30，31，34，42，48，60，61，62，63，64，65，66，67，68，69，72，73，74，75，76，81，93，99强0.5880.2070.705
2421，11，12，16，17，20，21，22，23，24，26，27，28，32，35，39，40，41，45，46，47，49，51，53，56，57，58，59，70，71，77，78，79，83，84，85，86，88，89，91，97，98一般0.4990.0260.687
3242，4，6，7，10，18，33，36，37，38，43，44，50，52，54，55，80，82，87，90，92，94，95，96弱0.6310.2650.768

Fig.5

Variations of turn-light status with driving distance when participants dealing with first takeover event"

Fig.6

Structure of FA-LST model for recognizing driving intention"

Table 11

Confusion matrices of driving intention recognition obtained by LSTM and FA-LST models"

真实意图制动左转左转直行制动右转制动右转
FA-LSTLSTMFA-LSTLSTMFA-LSTLSTMFA-LSTLSTMFA-LSTLSTMFA-LSTLSTM
预测意图制动左转8534227101010000
左转113723151067000000
直行000212331226121500
制动00001225133133594600
右转0000320045845900
制动右转000000009900

Table 12

Confusion matrices of driving intention recognition obtained by SVM and FA-SVM models"

真实意图制动左转左转直行制动右转制动右转
FA-SVMSVMFA-SVMSVMFA-SVMSVMFA-SVMSVMFA-SVMSVMFA-SVMSVM
预测意图制动左转706911954480011
左转993112964254244500
直行305161124812411310374000
制动864818181081060011
右转0071040562139637811
制动右转000000000033

Table 13

Performance indices of driving intention recognition"

性能指标模型驾驶意图
制动左转左转直行制动右转制动右转
精确率FA-LST0.9880.9440.9800.9920.8690.000
LSTM0.9710.8120.9280.9780.8840.000
FA-SVM0.7780.8100.9220.8370.9060.500
SVM0.8210.7710.9040.8220.8940.500
召回率FA-LST0.7940.9710.9980.6520.9930.000
LSTM0.3180.8220.9930.6520.9960.000
FA-SVM0.7690.8450.9230.7770.8881.000
SVM0.7580.8040.9180.7630.8481.000
F1-分数FA-LST0.8800.9570.9890.7870.9270.000
LSTM0.4790.8170.9590.7820.9370.000
FA-SVM0.7730.8270.9220.8060.8970.667
SVM0.7880.7870.9110.7910.8700.667
准确率FA-LST0.951
LSTM0.903
FA-SVM0.890
SVM0.872
1 de LaTorre G, Rad P, Choo K R. Driverless vehicle security: challenges and future research opportunities[J]. Future Generation Computer Systems, 2020, 108: 1092-1111.
2 SAE J3016—2016. Taxonomy and definitions for terms related to driving automation systems for on-road motor vehicles [S].
3 SAE J3016—2018. Taxonomy and definitions for terms related to driving automation systems for on-road motor vehicles [S].
4 National Highway Traffic Safety Administration. ODI resume[R]. Washington, DC: National Highway Traffic Safety Administration, 2017.
5 Lu Z J, Coster X, de Winter J C F. How much time do drivers need to obtain situation awareness? a laboratory-based study of automated driving[J]. Applied Ergonomics, 2017, 60: 293-304.
6 Endsley M R. Situation awareness global assessment technique (SAGAT)[C]∥Proceedings of the IEEE 1988 National Aerospace and Electronics Conference, Dayton, OH, USA, 1988: 789-795.
7 Samuel S, Borowsky A, Zilberstein S, et al. Minimum time to situation awareness in scenarios involving transfer of control from an automated driving suite[J]. Transportation Research Record, 2016, 2602(1): 115-120.
8 Wright T J, Samuel S, Borowsky A, et al. Experienced drivers are quicker to achieve situation awareness than inexperienced drivers in situations of transfer of control within a level 3 autonomous environment[C]∥Proceedings of the Human Factors and Ergonomics Society Annual Meeting, Los Angeles, CA, USA, 2016: 270-273.
9 Vlakveld W, van Nes N, de Bruin J, et al. Situation awareness increases when drivers have more time to take over the wheel in a level 3 automated car: a simulator study[J]. Transportation Research Part F: Psychology and Behaviour, 2018, 58: 917-929.
10 Petermeijer S M, Bazilinskyy P, Bengler K, et al. Take-over again: investigating multimodal and directional tors to get the driver back into the loop[J]. Applied Ergonomics, 2017, 62: 204-215.
11 Forster Y, Naujoks F, Neukum A, et al. Driver compliance to take-over requests with different auditory outputs in conditional automation[J]. Accident Analysis and Prevention, 2017, 109: 18-28.
12 Radlmayr J, Fischer F M, Bengler K. The influence of non-driving related tasks on driver availability in the context of conditionally automated driving[J]. Proceedings of the 20th Congress of the International Ergonomics Association, 2018, 823: 295-304.
13 Yoon S H, Ji Y G. Non-driving-related tasks, workload, and takeover performance in highly automated driving contexts[J]. Transportation Research Part F: Psychology and Behaviour, 2019, 60: 620-631.
14 Gold C, Körber M, Lechner D, et al. Taking over control from highly automated vehicles in complex traffic situations: the role of traffic density[J]. Human Factors, 2016, 58(4): 642-652.
15 Roche F, Thüring M, Trukenbrod A K. What happens when drivers of automated vehicles take over control in critical brake situations?[J]. Accident Analysis and Prevention, 2020, 144: No.105588.
16 Clark H, Feng J. Age differences in the takeover of vehicle control and engagement in non-driving-related activities in simulated driving with conditional automation[J]. Accident Analysis and Prevention, 2017, 106: 468-479.
17 Epple S, Roche F, Brandenburg S. The sooner the better: drivers' reactions to two-step take-over requests in highly automated driving[C]∥Proceedings of the Human Factors and Ergonomics Society Annual Meeting, Los Angeles, CA, USA, 2018, 62(1): 1883-1887.
18 Wiedemann K, Naujoks F, Wörle J, et al. Effect of different alcohol levels on take-over performance in conditionally automated driving[J]. Accident Analysis and Prevention, 2018, 115: 89-97.
19 Vogelpohl T, Kühn M, Hummel T, et al. Asleep at the automated wheel: sleepiness and fatigue during highly automated driving[J]. Accident Analysis and Prevention, 2019, 126: 70-84.
20 王海星, 王翔宇, 王招贤, 等. 基于数据挖掘的危险货物运输风险驾驶行为聚类分析[J]. 交通运输系统工程与信息, 2020, 20(1): 183-189.
Wang Hai-xing, Wang Xiang-yu, Wang Zhao-xian, et al. Dangerous driving behavior clustering analysis for hazardous materials transportation based on data mining[J]. Journal of Transportation Systems Engineering and Information Technology, 2020, 20(1): 183-189.
21 Zhou T, Zhang J. Analysis of commercial truck drivers' potentially dangerous driving behaviors based on 11-month digital tachograph data and multilevel modeling approach[J]. Accident Analysis and Prevention, 2019, 132: No.105256.
22 孙剑, 张一豪, 王俊骅. 基于自然驾驶数据的分心驾驶行为识别方法[J]. 中国公路学报, 2020, 33(9): 225-235.
Sun Jian, Zhang Yi-hao, Wang Jun-hua. Detecting distraction behavior of drivers using naturalistic driving data[J]. China Journal of Highway and Transport, 2020, 33(9): 225-235.
23 Jia S, Hui F, Li S N, et al. Long short-term memory and convolutional neural network for abnormal driving behaviour recognition[J]. IET Intelligent Transport Systems, 2020, 14(5): 306-312.
24 张姝玮, 郭忠印, 杨轸, 等. 驾驶行为多重分形特征在驾驶疲劳检测中的应用[J]. 吉林大学学报: 工学版, 2021, 51(2): 557-564.
Zhang Shu-wei, Guo Zhong-yin, Yang Zhen, et al. Application of multi-fractal features of driving performance in driver fatigue detection[J]. Journal of Jilin University (Engineering and Technology Edition), 2021, 51(2): 557-564.
25 庄明科, 白海峰, 谢晓非. 驾驶人员风险驾驶行为分析及相关因素研究[J]. 北京大学学报: 自然科学版, 2007, 4: 75-82.
Zhuang Ming-ke, Bai Hai-feng, Xie Xiao-fei. A study on risky driving behavior and related factors[J]. Acta Scientiarum Naturalium Universitatis Pekinensis (Natural Science Edition), 2007, 4: 75-82.
26 Child D. The Essentials of Factor Analysis[M]. London: Cassell Educational, 1990.
27 Kaiser H F. The varimax criterion for analytic rotation in factor analysis[J]. Psychometrika, 1958, 23 (3): 187-200.
28 Macqueen J. Some methods for classification and analysis of multivariate observations[C]∥Proceedings of Berkeley Symposium on Mathematical Statistics and Probability, Berkeley, CA, USA, 1967: 281-297.
29 Sun H D, Chen Y Y, Lai J H, et al. Identifying tourists and locals by K-means clustering method from mobile phone signaling data[J]. Journal of Transportation Engineering, Part A: Systems, 2021, 147(10): No.04021070.
30 Hochreiter S, Schmidhuber J. Long short-term memory[J]. Neural Computation, 1997, 9(8): 1735-1780.
31 赵申剑, 黎彧君, 符天凡, 等 译. 深度学习[M]. 北京: 人民邮电出版社, 2017.
[1] Xue XIAO,Ke-ping LI,Bo PENG,Man-wei CHANG. Integrated lane⁃changing model of decision making and motion planning for autonomous vehicles [J]. Journal of Jilin University(Engineering and Technology Edition), 2023, 53(3): 746-757.
[2] Min MA,Da-wei HU,Lan SHU,Zhuang-lin MA. Resilience assessment and recovery strategy on urban rail transit network [J]. Journal of Jilin University(Engineering and Technology Edition), 2023, 53(2): 396-404.
[3] Jing WANG,Feng WAN,Chun-jiao DONG,Chun-fu SHAO. Modelling on catchment area and attraction intensity of urban rail transit stations [J]. Journal of Jilin University(Engineering and Technology Edition), 2023, 53(2): 439-447.
[4] Song FANG,Jian-xiao MA,Gen LI,Ling-hong SHEN,Chu-bo XU. Traffic risk analysis of moving work zone on right lane of city expressway [J]. Journal of Jilin University(Engineering and Technology Edition), 2022, 52(8): 1786-1791.
[5] Song-xue GAI,Xiao-qing ZENG,Xiao-yuan YUE,Zi-hao YUAN. Parking guidance model based on user and system bi⁃level optimization algorithm [J]. Journal of Jilin University(Engineering and Technology Edition), 2022, 52(6): 1344-1352.
[6] Hong-feng XU,Hong-jin CHEN,Dong ZHANG,Qian-hui LU,Na AN,Xian-cai Geng. Fully⁃actuated signal timing technique for isolated signalized intersections in connected vehicle environment [J]. Journal of Jilin University(Engineering and Technology Edition), 2022, 52(6): 1324-1336.
[7] Guo-zhu CHENG,Qiu-yue SUN,Yue-bo LIU,Ji-long CHEN. Cut⁃in behavior model based on game theoretic approach on urban roads [J]. Journal of Jilin University(Engineering and Technology Edition), 2022, 52(12): 2839-2844.
[8] Heng-yan PAN,Wen-hui ZHANG,Bao-yu HU,Zun-yan LIU,Yong-gang WANG,Xiao ZHANG. Construction and robustness analysis of urban weighted subway⁃bus composite network [J]. Journal of Jilin University(Engineering and Technology Edition), 2022, 52(11): 2582-2591.
[9] Zhuang-lin MA,Shan-shan CUI,Da-wei HU. Urban residents' low⁃carbon travel intention after implementation of driving restriction policy [J]. Journal of Jilin University(Engineering and Technology Edition), 2022, 52(11): 2607-2617.
[10] Jing-xian WU,Hua-peng SHEN,Yin HAN,Min YANG. Residents' commuting time model under the nonlinear impact of urban built environment [J]. Journal of Jilin University(Engineering and Technology Edition), 2022, 52(11): 2568-2573.
[11] Jie MA,Jia-jun HUANG,Jun TIAN,Yang-hui DONG. Simulation modeling of pedestrian target decision⁃making in evacuation process in transfer corridors of subway stations [J]. Journal of Jilin University(Engineering and Technology Edition), 2022, 52(11): 2600-2606.
[12] Feng XUE,Chuan-lei HE,Qian HUANG,Jian LUO. Coordination degree of multimodal rail transit network [J]. Journal of Jilin University(Engineering and Technology Edition), 2021, 51(6): 2040-2050.
[13] Bo PENG,Yuan-yuan ZHANG,Yu-ting WANG,Ju TANG,Ji-ming XIE. Automatic traffic state recognition from videos based on auto⁃encoder and classifiers [J]. Journal of Jilin University(Engineering and Technology Edition), 2021, 51(3): 886-892.
[14] Dian-hai WANG,Xin-yi SHEN,Xiao-qin LUO,Sheng JIN. Offset optimization with minimum average vehicle delay [J]. Journal of Jilin University(Engineering and Technology Edition), 2021, 51(2): 511-523.
[15] Xian-min SONG,Ming-ye ZHANG,Zhen-jian LI,Xin WANG,Ya-nan ZHANG. Setting of dynamic bus lane and its simulation analysis and evaluation [J]. Journal of Jilin University(Engineering and Technology Edition), 2020, 50(5): 1677-1686.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
Copyright © Journal of Jilin University(Engineering and Technology Edition), All Rights Reserved.
Tel: +86-431-85095297　Fax: +86-431-85094074　E-mail: xbgxb@jlu.edu.cn
Powered by Beijing Magtech Co. Ltd