Journal of Jilin University(Engineering and Technology Edition) ›› 2026, Vol. 56 ›› Issue (7): 2006-2019.doi: 10.13229/j.cnki.jdxbgxb.20241283

Previous Articles    

A second order decision-making dynamic offloading method for vehicle edge computing tasks

Tao JU(),Wen-jin ZHANG,Yao YANG,Jiu-yuan HUO   

  1. School of Electronic and Information Engineering，Lanzhou Jiaotong University，Lanzhou 730070，China
  • Received:2024-11-29 Online:2026-07-01 Published:2026-08-12

RICH HTML

 0   

PDF (PC)

2

Abstract:

In response to the issue of low computational performance of edge computing systems when dealing with discrete-continuous hybrid decision-making problems in vehicular network edge computing scenarios， a distributed multi-stage decision-making strategy for dynamic offloading of vehicular network edge computing tasks based on deep reinforcement learning is proposed. By dividing the task processing procedure into two sub-problems： offloading decisions and resource allocation， an algorithm framework capable of handling both discrete and continuous action domains was designed based on the D3DQN network and TD3 network. In order to further enhance effective search in the action space and improve algorithm performance， an optimization and dynamic priority update mechanism for search in the action space was designed. Finally， based on the above algorithm framework and optimization mechanisms， a multi-stage decision-making algorithm for dynamic offloading of vehicle network edge computing tasks was developed and implemented. This algorithm can effectively solve discrete-continuous hybrid decision-making problems in vehicle network edge computing scenarios， utilize edge computing system resources， and ensure that vehicles can complete the offloading of computing tasks with minimal delay and energy consumption， according to the current network status and task size. Simulation results show that， compared with baseline offloading methods， the proposed method has faster convergence and lower latency energy consumption. It can make better use of the computing resources of the edge system to provide computing services for vehicle edge task requests， thus further enhancing the quality of service for users.

Key words: internet of vehicles, mobile edge computing, deep reinforcement learning, dynamic task offloading, resource allocation

CLC Number: 

  • TP311

Fig.1

Multi-server multi-task vehicle task offloading system"

Fig.2

D3DQN algorithm framework"

Fig.3

TD3 algorithmic framework"

Fig.4

SODDO task offloading framework"

Fig.5

UCB work process"

Table 1

Experimental parameter"

参 数数值
车辆数量20
基站覆盖范围（直径）/m1 200
高斯白噪声N0/dBm-110
参考信道功率增益β0/dB-30
车辆最大计算资源fmax/ GHzIPC*4*1.2*3
任务复杂度C/（cycles·bit-1）［50，1 250］
任务最大时延T?/s｛0.75，1.5｝
路径损耗指数θ4
车辆行驶速度/（km·h-1）50
单RSU覆盖范围（直径）/m400
无线通信带宽B/ MHz5
传输功率p/ mW100
VEC最大计算资源Fmax/GHzIPC*32*1.2*4
任务大小D/kB［100，1 000］
单位CPU周期能耗κ10-28
批处理尺寸batchsize64

Fig.6

SODDO convergence at different learning rates"

Fig.7

Convergence performance of SODDO under different discount factors"

Fig.8

Convergence performance of SODDO under different weights"

Fig.9

Convergence performance of different algorithms"

Fig.10

Total execution latency comparison"

Fig.11

Total transmission delay comparison"

Fig.12

Total latency comparison"

Fig.13

Comparison of total energy consumption"

Fig.14

Comparison of latency and energy consumption reduction rates"

Fig.15

Comparison of algorithm performance with different number of vehicles"

Fig.16

Comparison of algorithm performance with different number of RSUs"

[1] Sharma S, Kaushik B. A survey on internet of vehicles: applications, security issues & solutions[J]. Vehicular Communications, 2019, 20:100182.
[2] 况博裕, 李雨泽, 顾芳铭,等. 车联网安全研究综述: 威胁、对策与未来展望[J]. 计算机研究与发展, 2023, 60(10): 2304-2321.
Kuang Bo-yu, Li Yu-ze, Gu Fang-ming, et al. Review of internet of vehicle security research: threats, countermeasures, and future prospects[J]. Journal of Computer Research and Development, 2023, 60(10): 2304-2321.
[3] Boban M, Kousaridas A, Manolakis K, et al. Connected roads of the future: use cases, requirements, and design considerations for vehicle-to-everything communications[J]. IEEE Vehicular Technology Magazine, 2018, 13(3): 110-123.
[4] Yang D G, Jiang K, Zhao D, et al. Intelligent and connected vehicles: current status and future perspectives[J]. Science China Technological Sciences, 2018, 61: 1446-1471.
[5] Zhang W Y, Zhang Z J, chao H C. Cooperative fog computing for dealing with big data in the internet of vehicles: architecture and hierarchical resource management[J]. IEEE Communications Magazine, 2017, 55(12): 60-67.
[6] Jaehwan L, Woongsoo N. A survey on vehicular edge computing architectures[C]∥2022 13th International Conference on Information and Communication Technology Convergence(ICTC), Jeju, Island, Korea, 2022: 2198-2200.
[7] Mao Y Y, You C S, Zhang J, et al. A survey on mobile edge computing: the communication perspective[J]. IEEE Communications Surveys & Tutorials, 2017, 19(4): 2322-2358.
[8] Tang M, Wong V W S. Deep reinforcement learning for task offloading in mobile edge computing systems[J]. IEEE Transactions on Mobile Computing, 2022, 21(6): 1985-1997.
[9] Zhao N, Liang Y C, Niyato D, et al. Deep reinforcement Learning for user association and resource allocation in heterogeneous cellular networks[J]. IEEE Transactions on Wireless Communications, 2019, 18(11): 5141-5152.
[10] Dai Y Y, Xu D, Zhang K, et al. Deep reinforcement learning for edge computing and resource allocation in 5G beyond[C]∥2019 IEEE 19th International Conference on Communication Technology, Xi'an, China, 2019: 866-870.
[11] Raeisi-varzaneh M, Dakkak O, Habbal A, et al. Resource scheduling in edge computing: architecture, taxonomy, open issues and future research directions[J]. IEEE Access, 2023, 11: 25329-25350.
[12] Feng J, Yu F R, Pei Q Q, et al. Joint optimization of radio and computational resources allocation in blockchain-enabled mobile edge computing systems[J]. IEEE Transactions on Wireless Communications, 2020, 19(6): 4321-4334.
[13] 李智勇, 王琦, 陈一凡, 等. 车辆边缘计算环境下任务卸载研究综述[J]. 计算机学报, 2021, 44(5): 963-982.
Li Zhi-yong, Wang Qi, Chen Yi-fan, et al. A survey on task offloading research in vehicular edge computing[J]. Chinese Journal of Computers, 2021, 44(5): 963-982.
[14] 许斌, 赵云凯, 朱剑鸣, 等. 移动边缘计算不确定性任务持续卸载及资源分配方法[J]. 软件学报, 2023, 35(3): 1466-1484.
Xu Bin, Zhao Yun-kai, Zhu Jian-ming, et al. Continuous offloading and resource allocation method of uncertain tasks in mobile edge computing[J]. Journal of Software, 2023, 35(3): 1466-1484.
[15] Liu G Z, Dai F, Huang B, et al. A collaborative computation and dependency-aware task offloading method for vehicular edge computing: a reinforcement learning approach[J]. Journal of Cloud Computing, 2022, 11(1): 1-15.
[16] 孙彦景, 余政达, 陈瑞瑞, 等. 车联网中基于深度强化学习的高可靠资源分配算法[J]. 重庆邮电大学学报: 自然科学版, 2023, 35(4): 706-714.
Sun Yan-jing, Yu Zheng-da, Chen Rui-rui, et al. Deep reinforcement learning based high reliability resource allocation algorithm for internet of vehicles[J]. Journal of Chongqing University of Posts and Telecommunications(Natural Science Edition), 2023, 35(4): 706-714.
[17] Hao H, Ding W, Zhang W. Time-continuous computing offloading algorithm with user fairness guarantee[J]. Journal of Network and Computer Applications, 2024, 223: 103826.
[18] Yan J, Bi S Z, Zhang Y J A. Offloading and resource allocation with general task graph in mobile edge computing: a deep reinforcement learning approach[J]. IEEE Transactions on Wireless Communications, 2020, 19(8): 5404-5419.
[19] Zhao J H, Li Q P, Gong Y, et al. Computation offloading and resource allocation for cloud assisted mobile edge computing in vehicular networks[J]. IEEE Transactions on Vehicular Technology, 2019, 68(8): 7944-7956.
[20] Han H, Wu D G, Zhou F H, et al. Dynamic task offloading in MEC-enabled IoT networks: a hybrid DDPG-D3QN approach[C]∥2021 IEEE Global Communications Conference(GLOBECOM). Madrid, Spain, 2021: 1-6.
[21] Guan T, Yao Y S, Chao Y, et al. Energy-efficient computing offloading policy for MEC-assisted power sensor network: a D3QN approach[C]∥2023 International Conference on Cyber-Physical Social Intelligence(ICCSI), Xi'an, China, 2023: 180-185.
[22] Wang Z Y, Schaul T, Hessel M, et al. Dueling Network Architectures for Deep reinforcement learning[C]∥JMLR.org. Proceedings of the 33rd International Conference on International Conference on Machine Learning-Volume 48, New York, NY, USA, 2016: 1995-2003.
[23] Hasselt H V, Guez A, Silver D. Deep reinforcement learning with double Q-learning[C]∥AAAI'16. Proceeding of the Thirtieth AAAI Conference on Artificial Intelligence, Phoenix, Arizona, 2016: 2094-2100.
[24] 巨涛, 王志强, 刘帅, 等. D3DQN-CAA: 一种基于DRL的自适应边缘计算任务调度方法[J].湖南大学学报: 自然科学版, 2024, 51(6): 73-85.
Ju Tao, Wang Zhi-qiang, Liu Shuai, et al. D3DQN-CAA:a DRL-based adaptive edge computing task scheduling method[J]. Journal of Hunan University(Natural Sciences), 2024, 51(6): 73-85.
[25] 张红霞, 吕智豪, 席诗语, 等. 面向绿色计算的车辆协同任务卸载方法[J]. 电子与信息学报, 2024, 48(1): 175-183.
Zhang Hong-Xia, Lv Zhi-hao, Xi Shi-yu, et al. A method for offloading vehicle collaborative tasks for green computing[J]. Journal of Electronics & Information Technology, 2024, 48(1): 175-183.
[26] Kiran B R, Sobh I, Talpaert V, et al. Deep reinforcement learning for autonomous driving: A survey[J]. IEEE Transactions on Intelligent Transportation Systems, 2022, 23(6): 4909-4926.
[27] Zhou D R, Li L H, Gu Q Q. Neural contextual bandits with ucb-based exploration[C]∥International Conference on Machine Learning. 2020: 11492-11502.
[28] Zhong J W, Chen X, Jiao L B. A fair energy-efficient offloading scheme for heterogeneous fog-enabled IoT using D3QN[C]∥2023 25th International Conference on Communication Software and Networks(ICCSN), IEEE, 2023: 47-52.
[29] Min M H, Xiao L, Chen Y, et.al. Learning-based computation offloading for IoT devices with energy harvesting[J]. IEEE Transactions on Vehicular Technology, 2019, 68(2): 1930-1941.
[30] Geng L W, Zhao H B, Wang J Y, et al. Deep-reinforcement-learning-based distributed computation offloading in vehicular edge computing networks[J]. IEEE Internet of Things Journal, 2023, 10(14): 12416-12433.
[31] 郭晓东, 郝思达, 王丽芳. 基于深度强化学习的车辆边缘计算任务卸载方法[J]. 计算机应用技术, 2023, 40(9): 2803-2807, 2814.
Guo Xiao-dong, Hao Si-da, Wang Li-fang. Task offloading method based on deep reinforcement learning for vehicular edge computing[J]. Application Research of Computers, 2023, 40(9): 2803-2807, 2814.
[32] 邝祝芳, 陈清林, 李林峰, 等. 基于深度强化学习的多用户边缘计算任务卸载调度与资源分配算法[J]. 计算机学报, 2022, 45(4): 812-824.
Kuang Zhu-fang, Chen Qing-li, Li Li-feng, et al. Multi-user edge computing task offloading scheduling and resource allocation based on deep reinforcement learning[J]. Chinese Journal of Computers, 2022, 45(4): 812-824.
[33] 许小龙, 方子介, 齐连永, 等. 车联网边缘计算环境下基于深度强化学习的分布式服务卸载方法[J]. 计算机学报, 2021, 44(12): 2382-2405.
Xu Xiao-long, Fang Zi-jie, Qi Lian-yong, et al. A deep reinforcement learning-based distributed service offloading method for edge computing empowered internet of vehicles[J]. Chinese Journal of Computers, 2021, 44(12): 2382-2405.
[1] Cheng-jun TIAN,Yu YAN,Ren-wei CUI,Jin-tong ZHANG. Autonomous grasping algorithm of robotic arm based on deep reinforcement learning [J]. Journal of Jilin University(Engineering and Technology Edition), 2026, 56(3): 662-669.
[2] Xin-chun LI,He-yuan SUN,Chi XU. Network resource allocation method of aerospace edge computing based on deep Q network algorithm [J]. Journal of Jilin University(Engineering and Technology Edition), 2025, 55(7): 2418-2424.
[3] Jian WANG,Chen-wei JIA. Trajectory prediction model for intelligent connected vehicle [J]. Journal of Jilin University(Engineering and Technology Edition), 2025, 55(6): 1963-1972.
[4] Zi-hao SHEN,Yong-sheng GAO,Hui WANG,Pei-qian LIU,Kun LIU. Deep deterministic policy gradient caching method for privacy protection in Internet of Vehicles [J]. Journal of Jilin University(Engineering and Technology Edition), 2025, 55(5): 1638-1647.
[5] Shu-xu ZHAO,Zhi-chao SUN,Xiao-long WANG. Dynamic authentication protocol for mobile edge computing scenarios [J]. Journal of Jilin University(Engineering and Technology Edition), 2025, 55(3): 1050-1060.
[6] Yao-ping ZENG,Yu-ting XIA,Shi-sen CHEN,Yue-qiang LIU,Wei-wei JIANG. Energy⁃efficient offloading strategy research for multi⁃UAV assisted communication [J]. Journal of Jilin University(Engineering and Technology Edition), 2025, 55(12): 4083-4092.
[7] Wei-chao HU,Zhen-ming YANG,Peng-cheng YU,Yan-yan CHEN,She-qiang MA. Modeling interaction policy of autonomous vehicle and pedestrian based on deep reinforcement learning [J]. Journal of Jilin University(Engineering and Technology Edition), 2025, 55(10): 3180-3188.
[8] Han-ying HUANG,Peng-fei LI. Method and experiments on edge computing resource allocation in smart fishery [J]. Journal of Jilin University(Engineering and Technology Edition), 2025, 55(1): 316-324.
[9] Guang-he ZHU,Zhi-qiang ZHU,Yi-ping YUAN. Deep reinforcement learning optimization scheduling algorithm for continuous production line [J]. Journal of Jilin University(Engineering and Technology Edition), 2024, 54(7): 2086-2092.
[10] Jing-peng GAO,Guo-xuan WANG,Lu GAO. LSTM⁃MADDPG multi⁃agent cooperative decision algorithm based on asynchronous collaborative update [J]. Journal of Jilin University(Engineering and Technology Edition), 2024, 54(3): 797-806.
[11] Si-feng ZHU,Jiang-hao CAI,Zheng-yi CHAI,En-lin SUN. Computing offloading optimization scheme based on immune algorithm in edge computing scenes of internet of vehicles [J]. Journal of Jilin University(Engineering and Technology Edition), 2024, 54(1): 221-231.
[12] Jian ZHANG,Qing-yang LI,Dan LI,Xia JIANG,Yan-hong LEI,Ya-ping JI. Merging guidance of exclusive lanes for connected and autonomous vehicles based on deep reinforcement learning [J]. Journal of Jilin University(Engineering and Technology Edition), 2023, 53(9): 2508-2518.
[13] Yu-lin CHANG,Wen-qian XU,Chao SUN,Peng ZHANG. Day⁃to⁃day equilibrium of hybrid traffic considering obedience degree under internet of vehicles environment [J]. Journal of Jilin University(Engineering and Technology Edition), 2023, 53(4): 1085-1093.
[14] Yan-tao TIAN,Yan-shi JI,Huan CHANG,Bo XIE. Deep reinforcement learning augmented decision⁃making model for intelligent driving vehicles [J]. Journal of Jilin University(Engineering and Technology Edition), 2023, 53(3): 682-692.
[15] Wei-chao ZHUANG,Hao-nan Ding,Hao-xuan DONG,Guo-dong YIN,Xi WANG,Chao-bin ZHOU,Li-wei XU. Learning based eco⁃driving strategy of connected electric vehicle at signalized intersection [J]. Journal of Jilin University(Engineering and Technology Edition), 2023, 53(1): 82-93.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
Copyright © Journal of Jilin University(Engineering and Technology Edition), All Rights Reserved.
Tel: +86-431-85095297　Fax: +86-431-85094074　E-mail: xbgxb@jlu.edu.cn
Powered by Beijing Magtech Co. Ltd