Journal of Jilin University(Engineering and Technology Edition) ›› 2023, Vol. 53 ›› Issue (1): 82-93.doi: 10.13229/j.cnki.jdxbgxb20210598

Previous Articles     Next Articles

Learning based eco⁃driving strategy of connected electric vehicle at signalized intersection

Wei-chao ZHUANG(),Hao-nan Ding,Hao-xuan DONG,Guo-dong YIN(),Xi WANG,Chao-bin ZHOU,Li-wei XU   

  1. School of Mechanical Engineering，Southeast University，Nanjing 211189，China
  • Received:2021-06-26 Online:2023-01-01 Published:2023-07-23
  • Contact: Guo-dong YIN E-mail:wezhuang@seu.edu.cn;ygd@seu.edu.cn

RICH HTML

 12   

PDF (PC)

594

Abstract:

A deep reinforcement learning based eco-driving strategy for connected electric vehicle （EV） was proposed to improve its energy efficiency at signalized intersection. Firstly， the dynamics of the EV is modelled， and the simulation environment of signalized intersection crossing scenario is established. Secondly， the reward function including multiple objectives is designed considering energy consumption reduction and travel efficiency improvement. The Deep Determinate Policy Gradient （DDPG） is developed to control the vehicle acceleration in continuous action space. Finally， a Monte Carlo simulation is conducted to verify the effectiveness and robustness of proposed method in different driving conditions. The simulation results show that the proposed strategy can improve the vehicle energy efficiency while ensuring travel efficiency in both single and multiple intersection scenarios， compared to a conventional accelerate-constant-brake strategy. In addition， a field test is conducted based on a developed connected automated vehicle digital twin platform. The experiment results show that the proposed reinforcement learning based eco-driving strategy has the potential to improve the vehicle energy efficiency and travel efficiency， simultaneously.

Key words: vehicle engineering, connected electric vehicle, eco-driving, deep reinforcement learning, signalized intersection, digital twin

CLC Number: 

  • U469.72

Fig.1

Economic driving at signalized intersections"

Fig.2

Characteristics of motor efficiency and torque-speed"

Fig.3

RL based intersection approaching strategy"

Fig.4

Signal phase and state model"

Table 1

Environment state input"

输入量/单位描 述
v/（m·s-1）车辆当前车速
ve/（m·s-1）与建议车速的差值
∫Δvdt累计误差值
a/（m·s-2）当前时刻的加速度
lt/m当前时刻车辆到路口的距离
ΔE/kJ每一步电池消耗的能量
∫ΔEdt/kJ累计能量消耗
trem/s当前信号灯剩余时间
Tstate信号灯的状态

Fig.5

DDPG-based intersection approaching strategy framework"

Table 2

Parameter setting of DDPG algorithm"

参数数值参数数值
目标平滑更新因子10-3单次训练最大步数103
折扣系数γ0.99总仿真时间/s102
mini?batch32停止训练奖励值103
经验回放缓存10-6随机噪声方差0.6
采样时间t/s0.1随机噪声衰退率10-5

Fig.6

Accumulated reward changes during DDPG training"

Fig.7

Simulation results of different strategies"

Table 3

Comparison of energy consumption of different strategies"

策 略ACBDPDDPG
提升比例

DRL vs. ACB： 37.3%

DRL vs. DP： -2.7%

通过时间/s56.456.056.0
初始速度/（km·h-1）36.036.036.0
末端速度/（km·h-1）10.431.436.4
电池能耗/kJ291.4213.1230.3
动能变化/kJ-82.5-21.51.8
总能量/kJ373.9234.6228.5

Fig.8

Comparison of stochastic performance validation between DDPG and ACB strategy"

Table 4

Comparison of energy consumption"

策略通行时间/s能耗/kJ提升比例/%
ACB290.71751-
DRL289.9120131.4

Fig.9

Validation of multi-intersections"

Fig.10

Architecture of digital twin platform for CAV"

Fig.11

Experimental road selection"

Fig.12

Experimental results of digital-twin system"

1 《中国公路学报》编辑部.中国汽车工程学术研究综述·2017 [J]. 中国公路学报, 2017, 30(6): 1-197.
Editorial Department of China Journal of Highway. Review on China's automotive engineering research progress: 2017[J]. China Journal of Highway and Transport, 2017, 30(6): 1-197.
2 徐兴,陈特,陈龙,等.分布式驱动电动汽车转矩节能优化分配[J].中国公路学报, 2018, 31(5): 183-190.
Xu Xing, Chen Te, Chen Long, et al. Optimal distribution of torque energy saving for distributed drive electric vehicles[J]. China Journal of Highway and Transport, 2018, 31(5): 183-190.
3 Michael Sivak, Brandon Schoettle. Eco-driving: strategic, tactical, and operational decisions of the driver that influence vehicle fuel economy[J]. Transport Policy, 2012, 22: 96-99.
4 付锐,张雅丽,袁伟.生态驾驶研究现状及展望[J].中国公路学报, 2019, 32(3): 1-12.
Fu Rui, Zhang Ya-li, Yuan Wei. Research status and prospects of eco-driving[J]. China Journal of Highway and Transport, 2019, 32(3): 1-12.
5 Ye Fei, Hao Peng, Qi Xue-wei, et al. Prediction-based eco-approach and departure at signalized intersections with speed forecasting on preceding vehicles[J]. IEEE Transactions on Intelligent Transportation Systems, 2018, 20(4): 1378-1389.
6 金辉,张俊.智能车经济性起步车速规划研究[J]. 汽车工程, 2020, 42(2): 270-277.
Jin Hui, Zhang Jun. Research on economic speed planning of intelligent vehicle for starting stage[J]. Automotive Engineering, 2020, 42(2): 270-277.
7 洪金龙,高炳钊,董世营,等.智能网联汽车节能优化关键问题与研究进展[J]. 中国公路学报, 2021,34(11):306-334.
Hong Jin-long, Gao Bing-zhao, Dong Shi-ying, et al. Key problems and research progress of energy saving optimization for intelligent connected vehicles[J]. China Journal of Highway and Transport, 2021,34(11):306-334.
8 Rainer Stahlmann, Malte Möller, Alexej Brauer, et al. exploring GLOSA systems in the field: technical evaluation and results[J]. Computer Communications, 2018, 120: 112-124.
9 陈浩, 庄伟超, 殷国栋, 等.网联电动汽车信号灯控路口经济性驾驶策略[J]. 东南大学学报:自然科学版, 2021, 51(1): 178-186.
Chen Hao, Zhuang Wei-chao, Yin Guo-dong, et al. Eco-driving control strategy of connected electric vehicle at signalized intersection[J]. Journal of Southeast University (Natural Science Edition), 2021, 51(1): 178-186.
10 Hao Peng, Wu Guo-yuan, Kanok Boriboonsomsin, et al. Eco-approach and departure (EAD) application for actuated signals in real-world traffic[J]. IEEE Transactions on Intelligent Transportation Systems, 2018, 20(1): 30-40.
11 解少博,辛宗科,李会灵,等.插电式混合动力公交车电池配置和能量管理策略协同优化的研究[J]. 汽车工程, 2018, 40(6): 625-631, 645.
Xie Shao-bo, Xin Zong-ke, Li Hui-ling, et al. A study on coordinated optimization on battery capacity and energy management strategy for a plug-in hybrid electric bus[J]. Automotive Engineering, 2018, 40(6): 625-631, 645.
12 Li Chun-ming, Zhang Tao, Sun Xiao-xia, et al. Connected ecological cruise control strategy considering multi-intersection traffic flow[J]. IEEE Access, 2020, 8: 219378-219390.
13 Han Ji-hun, Ardalan Vahidi, Antonio Sciarretta. Fundamentals of energy efficient driving for combustion engine and electric vehicles: an optimal control perspective[J]. Automatica, 2019, 103: 558-572.
14 朱冰,蒋渊德,赵健,等.基于深度强化学习的车辆跟驰控制[J].中国公路学报,2019,32(6):53-60.
Zhu Bing, Jiang Yuan-de, Zhao Jian, et al. A car-following control algorithm based on deep reinforcement learning[J]. China Journal of Highway and Transport,2019,32(6):53-60.
15 Wu Yuan-kai, Tan Hua-chun, Peng Jian-kun, et al. Deep reinforcement learning of energy management with continuous control strategy and traffic information for a series-parallel plug-in hybrid electric bus[J]. Applied Energy, 2019, 247: 454-466.
16 Li Yue-cheng, He Hong-wen, Amir Khajepour, et al. Energy management for a power-split hybrid electric bus via deep reinforcement learning with terrain information[J]. Applied Energy, 2019, 255: 113762.
17 Zhou Mo-fan, Yu Yang, Qu Xiao-bo. Development of an efficient driving strategy for connected and automated vehicles at signalized intersections: a reinforcement learning approach[J]. IEEE Transactions on Intelligent Transportation Systems, 2019, 21(1): 433-443.
18 Zhu Zhao-xuan, Shobhit Gupta, Abhishek Gupta, et al. A deep reinforcement learning framework for eco-driving in connected and automated hybrid electric vehicles[J/OL].[2021-01-13]. .
19 Dimitri Bertsekas. Reinforcement Learning and Optimal Control[M]. Belmont: Athena Scientific, 2019.
20 Xu Xin, Zuo Lei, Li Xin, et al. A reinforcement learning approach to autonomous decision making of intelligent vehicles on highways[J]. IEEE Transactions on Systems, Man, and Cybernetics: Systems, 2018, 50(10): 3884-3897.
21 Pi Jian-zong. A reinforcement learning framework for autonomous eco-driving[D]. The Ohio State University, 2020.
22 John Schulman, Filip Wolski, Prafulla Dhariwal, et al. Proximal policy optimization algorithms[J/OL].[2017-04-23]..
23 David Silver, Guy Lever, Nicolas Heess, et al. Deterministic policy gradient algorithms[C]∥ International Conference on Machine Learning, Detroit, USA, 2014: 387-395.
[1] Lei CHEN,Yang WANG,Zhi-sheng DONG,Ya-qi SONG. A vehicle agility control strategy based on steering intent [J]. Journal of Jilin University(Engineering and Technology Edition), 2023, 53(5): 1257-1263.
[2] Xin CHEN,Guan-chen ZHANG,Kang-ming ZHAO,Jia-ning WANG,Li-fei YANG,De-rong SITU. Influence of lap welds on the lightweight design of welded aluminum structures [J]. Journal of Jilin University(Engineering and Technology Edition), 2023, 53(5): 1282-1288.
[3] Yong ZHANG,Feng-zhao MAO,Shui-chang LIU,Qing-yu WANG,Shen-gong PAN,Guang-sheng ZENG. Optimization on distortion grid of vehicle external flow field based on Laplacian Algorithm [J]. Journal of Jilin University(Engineering and Technology Edition), 2023, 53(5): 1289-1296.
[4] Shao-hua WANG,Kun CHU,De-hua SHI,Chun-fang YIN,Chun LI. Robust compound coordinated control of HEV based on finite⁃time extended state observation [J]. Journal of Jilin University(Engineering and Technology Edition), 2023, 53(5): 1272-1281.
[5] Yan-li YIN,Xue-jiang HUANG,Xiao-liang PAN,Li-tuan WANG,Sen ZHAN,Xin-xin ZHANG. Hierarchical control of hybrid electric vehicle platooning based on PID and Q⁃Learning algorithm [J]. Journal of Jilin University(Engineering and Technology Edition), 2023, 53(5): 1481-1489.
[6] Gui-shen YU,Xin CHEN,Zi-tao WU,Yi-xiong CHEN,Guan-chen ZHANG. Analysis of microstructure and mechanical properties of probeless friction stir spot welding joint in AA6061⁃T6 aluminum thin plate [J]. Journal of Jilin University(Engineering and Technology Edition), 2023, 53(5): 1338-1344.
[7] Yan-tao TIAN,Xing HUANG,Hui-qiu LU,Kai-ge WANG,Fu-qiang XU. Multi⁃mode behavior trajectory prediction of surrounding vehicle based on attention and depth interaction [J]. Journal of Jilin University(Engineering and Technology Edition), 2023, 53(5): 1474-1480.
[8] Hong-bo YANG,Wen-ku SHI,Zhi-yong CHEN,Nian-cheng GUO,Yan-yan ZHAO. Multi⁃objective optimization of macro parameters of helical gear based on NSGA⁃Ⅱ [J]. Journal of Jilin University(Engineering and Technology Edition), 2023, 53(4): 1007-1018.
[9] Rui ZHAO,Yun LI,Hong-yu HU,Zhen-hai GAO. Vehicle collision warning method at intersection based on V2I communication [J]. Journal of Jilin University(Engineering and Technology Edition), 2023, 53(4): 1019-1029.
[10] Xiao-bo CHEN,Ling CHEN. Variational Bayesian cooperative target tracking with unknown localization noise statistics [J]. Journal of Jilin University(Engineering and Technology Edition), 2023, 53(4): 1030-1039.
[11] Yan-tao TIAN,Yan-shi JI,Huan CHANG,Bo XIE. Deep reinforcement learning augmented decision⁃making model for intelligent driving vehicles [J]. Journal of Jilin University(Engineering and Technology Edition), 2023, 53(3): 682-692.
[12] Jian ZHANG,Jin-bo LIU,Yuan GAO,Meng-ke LIU,Zhen-hai GAO,Bin YANG. Localization algorithm of vehicular sensor based on multi⁃mode interaction [J]. Journal of Jilin University(Engineering and Technology Edition), 2023, 53(3): 772-780.
[13] Ke HE,Hai-tao DING,Xuan-qi LAI,Nan XU,Kong-hui GUO. Wheel odometry error prediction model based on transformer [J]. Journal of Jilin University(Engineering and Technology Edition), 2023, 53(3): 653-662.
[14] Yan-ran LIU,Qing-yu MENG,Hong-yan GUO,Jia-lin LI. Vehicle trajectory prediction combined with high definition map in graph attention mode [J]. Journal of Jilin University(Engineering and Technology Edition), 2023, 53(3): 792-801.
[15] Bo XIE,Rong GAO,Fu-qiang XU,Yan-tao TIAN. Stability control of human⁃vehicle shared steering system under low adhesion road conditions [J]. Journal of Jilin University(Engineering and Technology Edition), 2023, 53(3): 713-725.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
Copyright © Journal of Jilin University(Engineering and Technology Edition), All Rights Reserved.
Tel: +86-431-85095297　Fax: +86-431-85094074　E-mail: xbgxb@jlu.edu.cn
Powered by Beijing Magtech Co. Ltd