汽车纵向自动驾驶的因果推理型决策

doi:10.13229/j.cnki.jdxbgxb20190078

Abstract

Abstract:

In order to solve cause-and-effect problems during decision-making, in this paper, the Markov Decision Process (MDP) model is established by the analysis of car-following at first. Then, the state set and the action set are designed through the combination of driving simulator experimental data and driving risk principle. Third, the reward functions are designed according to different driving states. Finally, a causal reasoning mechanism during the process of decision-making is proposed and reinforcement Q-learning algorithm is applied to solve the MDP model. The feasibility and effectiveness of the proposed method are verified through the simulation tests with random driving conditions.

Key words: vehicle engineering, automatic driving, decision-making algorithm, Markov decision process, reinforcement Q learning

CLC Number:

U462.1

Zhen-hai GAO,Tian-jun SUN,Lei HE. Causal reasoning decision⁃making for vehicle longitudinal automatic driving[J].Journal of Jilin University(Engineering and Technology Edition), 2019, 49(5): 1392-1404.

Figures/Tables 19

Fig.1

Fig.2

Table 1

Elements of MDP model"

元素名称	符号
状态集	S
动作集	A
状态转移概率	P
阻尼系数(折扣因子)	$γ$
回报函数	R

Table 1

Fig.3

Fig.4

Fig.5

Fig.6

Table 2

Fig.7

Fig.8

Fig.9

Fig.10

Fig.11

Fig.12

Fig.13

Fig.14

Fig.15

Fig.16

Fig.17

References 23

1	宋威龙 . 城区动态环境下智能车辆行为决策研究[D]. 北京: 北京理工大学机械与车辆学院, 2016.
	Song Wei-long . Research on behavioral decision making for intelligent vehicles in dynamic urban environments[D]. Beijing: School of Mechanical and Vehicle Engineering, Beijing Institute of Technology, 2016.
2	王竣 . 自适应巡航控制系统多目标感知决策控制算法研究[D]. 长春: 吉林大学汽车工程学院, 2015.
	Wang Jun . Research on multi-objective perception-decision-control algorithm for adaptive cruise control systems[D]. Changchun: College of Automotive Engineering, Jilin University, 2015.
3	You C X , Lu J B , Filev D , et al . Highway traffic modeling and decision making for autonomous vehicle using reinforcement learning[C]⫽2018 IEEE Intelligent Vehicles Symposium(IV), Changshu, Suzhou, China , 2018: 1227-1232.
4	左磊 . 智能驾驶车辆自主决策与规划的增强学习方法研究[D]. 长沙: 国防科学技术大学研究生院, 2016.
	Zuo Lei . Autonomous driving decision making and planning for intelligent vehicles using reinforcement learning[D]. Changsha: School of Graduate, National University of Defense Technology, 2016.
5	郑睿 . 基于增强学习的无人车辆智能决策方法研究[D]. 长沙: 国防科技大学研究生院, 2013.
	Zheng Rui . Reinforcement learning-based intelligent decision-making methods for unmanned vehicles[D]. Changsha: School of Graduate, National University of Defense Technology, 2013.
6	刘春明 . 基于增强学习和车辆动力学的高速公路自主驾驶研究[D]. 长沙: 国防科技大学研究生院, 2014.
	Liu Chun-ming . Research on autonomous driving on highway roads based on reinforcement learning and vehicle dynamics[D]. Changsha: School of Graduate, National University of Defense Technology, 2014.
7	Gao H B , Shi G Y , Xie G T , et al . Car-following method based on inverse reinforcement learning for autonomous vehicle decision-making[J]. International Journal of Advanced Robotic Systems, 2018, 15(6): 1-11.
8	Chong L , Abbas M M , Medina F A , et al . A rule-based neural network approach to model driver naturalistic behavior in traffic[J]. Transportation Research Part C, 2013, 32(4): 207-223.
9	王喆, 杨柏婷, 刘昕, 等 . 基于模糊聚类的驾驶决策判别[J]. 吉林大学学报: 工学版, 2015, 45(5): 1414-1419.
	Wang Zhe , Yang Bo-ting , Liu Xin , et al . Discriminant analysis of driving decisions based on fuzzy clustering[J]. Journal of Jilin University (Engineering and Technology Edition), 2015, 45(5): 1414-1419.
10	杨慧媛 . 基于增强学习的优化控制方法及其在移动机器人中的应用[D]. 长沙: 国防科学技术大学研究生院, 2014.
	Yang Hui-yuan . Reinforcement learning-based optimal control methods with applications to mobile robots[D]. Changsha: School of Graduate, National University of Defense Technology, 2014.
11	李冲 . 基于学习的智能交通预测与多路径规划研究[D]. 长沙: 中南大学信息科学与工程学院, 2014.
	Li Chong . Q learning based prediction and multiple paths planning in intelligent transportation systems[D]. Changsha: School of Information Science and Engineering, Central South University, 2014.
12	高振海, 严伟, 李红建, 等 . 汽车自适应巡航线性参变间距控制算法[J]. 吉林大学学报: 工学版, 2016, 46(4): 1023-1029.
	Gao Zhen-hai , Yan Wei , Li Hong-jian , et al . Design of linear parametrically varying headway control algorithm for ACC vehicles[J]. Journal of Jilin University (Engineering and Technology Edition), 2016, 46(4): 1023-1029.
13	Bellman R E . A Markov decision process[J]. Journal of Mathematical Mech, 1957（6）: 679-684.
14	Howard R . Dynamic programming and Markov processes[M]. Cambridge, Massachusetts: MIT Press, 1960.
15	郭茂祖, 刘扬, 黄梯云 . 加强学习主要算法的比较研究[J]. 计算机工程与应用, 2001, 37(21): 16-18.
	Guo Mao-zu , Liu Yang , Huang Ti-yun . Comparative study of the main reinforcement learning algorithms[J]. Computer Engineering and Application, 2001, 37(21): 16-18.
16	Qu T , Yu S Y , Shi Z Q , et al . Modeling driver’s car-following behavior based on hidden markov model and model predictive control: a cyber-physical system approach[C]⫽The 11th Asian Control Conference (ASCC), Gold Coast Convention Centre, Australia, 2017: 114-119.
17	Chen X , Zhai Y , Lu C , et al . A learning model for personalized adaptive cruise control[C]⫽IEEE Intelligent Vehicles Symposium (IV), Redondo Beach, CA, USA, 2017: 379-384.
18	Hester T , Lopes M , Stone P . Learning exploration strategies in model-based reinforcement learning[C]⫽IEEE the 12th International Conference in Autonomous Agents and Multi-agent Systems (AAMAS), St Paul, Minnesota, 2013: 1069-1076.
19	田苗 . 强化学习算法的研究与实验[D]. 西安: 西安电子科技大学控制科学与工程学院, 2014.
	Tian Miao . Study and experiment of reinforcement learning algorithm[D]. Xi’an: School of Control Science and Engineering, Xidian University, 2014.
20	Gao Z , Sun T , He L . A multi-mode control strategy for EV based on typical situation[C]⫽WCX™ 17: SAE World Congress Experience, Detroit Michigan, United States, 2017: 438-447.
21	徐亮, 张自力 . 基于MAS的驾驶行为决策模型的研究[J]．计算机工程与科学, 2010, 32(5)： 154-158.
	Xu Liang , Zhang Zi-li . Research on the Decision-Making Model of Driving Behavior based on MAS[J].Computer Engineering and Science, 2010, 32(5): 154-158.
22	Solomon D . Accidents on Main Rural Highways Related to Speed[R]. Washington DC: Bureau of Public Roads, 1964.
23	Rigolli M , Brady M . Towards a behavioral traffic monitoring system[C]⫽International Joint Conference on Autonomous Agents and Multi-Agent Systems, The Netherlands, 2005: 449-454.

Related Articles 15

[1]	Fang-wu MA,Lu HAN,Yang ZHOU,Shi-ying WANG,Yong-feng PU. Multi material optimal design of vehicle product using polylactic acid composites [J]. Journal of Jilin University(Engineering and Technology Edition), 2019, 49(5): 1385-1391.
[2]	Jing LI,Qiu⁃jun SHI,Peng LIU,Ya⁃wei HU. Neural network sliding mode control of commercial vehicle ABS based on longitudinal vehicle speed estimation [J]. Journal of Jilin University(Engineering and Technology Edition), 2019, 49(4): 1017-1025.
[3]	Shun YANG,Yuan⁃de JIANG,Jian WU,Hai⁃zhen LIU. Autonomous driving policy learning based on deep reinforcement learning and multi⁃type sensor data [J]. Journal of Jilin University(Engineering and Technology Edition), 2019, 49(4): 1026-1033.
[4]	Xin CHEN,Ming LI,Xin⁃jian RUAN,Ning WANG,Jia⁃ning WANG. Investigation of vortical structures in wake of Ahmed body by delayed detached⁃eddy simulation turbulence model using immersed boundary method [J]. Journal of Jilin University(Engineering and Technology Edition), 2019, 49(4): 1034-1042.
[5]	Hua ZHOU,Zhi⁃gang YANG,Hui ZHU. Aerodynamic calculation of MIRA model correlated with wind tunnel test [J]. Journal of Jilin University(Engineering and Technology Edition), 2019, 49(4): 1043-1053.
[6]	Ji⁃qing CHEN,Meng⁃meng LIU,Feng⁃chong LAN. Experiment on overcharge safety of NCM battery and battery pack [J]. Journal of Jilin University(Engineering and Technology Edition), 2019, 49(4): 1072-1080.
[7]	Run⁃dong LIU,Jun MAO,Yan⁃hong XI,Hong⁃yu ZHANG,Fei PENG. Pressure pulse on windbreak impacting by cross⁃wind coupling with high⁃speed trains passing each other [J]. Journal of Jilin University(Engineering and Technology Edition), 2019, 49(4): 1054-1062.
[8]	Guo⁃feng QIN,Jing⁃xin NA,Wen⁃long MU,Wei TAN,Jian⁃ze LUAN,Hao SHEN. Degradation failure of adhesively bonded CFRP/aluminum alloy subjected to high temperature environment [J]. Journal of Jilin University(Engineering and Technology Edition), 2019, 49(4): 1063-1071.
[9]	Li⁃bin ZHANG,Dao WU,Hong⁃ying SHAN,Xiang⁃jing DENG. Design of axle load self⁃adjusting system for multi⁃axle vehicle based on brake tester [J]. Journal of Jilin University(Engineering and Technology Edition), 2019, 49(4): 1081-1091.
[10]	Shou⁃tao LI,Qiu⁃yuan LI,Hui LIU,Hui DING,Yan⁃tao TIAN,Ding⁃li YU. Sliding mode variable structure strategy for vehicle stability control [J]. Journal of Jilin University(Engineering and Technology Edition), 2019, 49(4): 1288-1292.
[11]	Yuan-ning LIU,Shuai LIU,Xiao-dong ZHU,Guang HUO,Tong DING,Kuo ZHANG,Xue JIANG,Shu-jun GUO,Qi-xian ZHANG. Iris secondary recognition based on decision particle swarm optimization and stable texture [J]. Journal of Jilin University(Engineering and Technology Edition), 2019, 49(4): 1329-1338.
[12]	CHANG Cheng,SONG Chuan-xue,ZHANG Ya-ge,SHAO Yu-long,ZHOU Fang. Minimizing inverter capacity of doubly-fed machine driving electric vehicles [J]. Journal of Jilin University(Engineering and Technology Edition), 2018, 48(6): 1629-1635.
[13]	XI Li-he,ZHANG Xin,SUN Chuan-yang,WANG Ze-xing,JIANG Tao. Adaptive energy management strategy for extended range electric vehicle [J]. Journal of Jilin University(Engineering and Technology Edition), 2018, 48(6): 1636-1644.
[14]	HE Ren,YANG Liu,HU Dong-hai. Design and analysis of refrigeration system supplied by solar auxiliary power of refrigerator car [J]. Journal of Jilin University(Engineering and Technology Edition), 2018, 48(6): 1645-1652.
[15]	NA Jing-xin,MU Wen-long,FAN Yi-sa,TAN Wei,YANG Jia-zhou. Effect of hygrothermal aging on steel-aluminum adhesive joints for automotive applications [J]. Journal of Jilin University(Engineering and Technology Edition), 2018, 48(6): 1653-1660.

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed

Comments

Recommended 0

No Suggested Reading articles found!

变量名称	符号
当前本车车速/(m·s^-1)	V_x
当前相对距离/m	D
当前相对速度/(m·s^-1)	V _r
当前本车加速度/(m·s^-2)	A_x
设定理想车速/(m·s^-1)	V _{_set}
设定理想距离/m	D _{_set}
下一时刻本车车速/(m·s^-1)	V_x ₁
下一时刻相对距离/m	D ₁
下一时刻相对速度/(m·s^-1)	V _r1
下一时刻本车加速度/(m·s^-2)	A

Causal reasoning decision⁃making for vehicle longitudinal automatic driving

RICH HTML

PDF (PC)