基于改进型DQN的无人机覆盖路径规划算法

doi:10.13229/j.cnki.jdxbgxb.20250237

摘要/Abstract

摘要：

针对无人机覆盖路径规划中传统Deep Q-network（DQN）存在的覆盖率低、路径冗余度高、收敛速度慢等问题，提出了一种基于多机制融合的改进型DQN模型。首先，将无人机的工作空间建模为二维栅格地图，进行预处理并设计奖励函数。其次，基于传统DQN，逐步融合Double DQN、Dueling network和Priority experience replay三种机制，运用独热编码等技术搭建多尺度卷积神经网络，构建了改进型DQN模型。最后，设计可视化仿真实验，并以栅格覆盖率等评价指标进行消融实验。实验结果表明：改进型DQN模型在覆盖路径规划任务中表现出色，其性能优于传统DQN等模型。相较于传统DQN，其栅格覆盖率提升了8.28%，栅格重复率、路径长度、碰撞障碍物次数、超出边界次数分别降低了28.44%、45.14%、77.31%、90.14%，且网络收敛速度显著加快。

关键词: 计算机应用, 深度强化学习, 覆盖路径规划, 无人机, 栅格地图

Abstract:

Aiming at the problems of low coverage， high path redundancy and slow convergence speed of traditional DQN in UAV coverage path planning， an improved DQN model based on multi-mechanism fusion was proposed. Firstly， the workspace of the UAV was modeled as a two-dimensional grid map. It was preprocessed and the reward function was designed. Secondly， based on the traditional DQN， three mechanisms of Double DQN， Dueling Network and Priority Experience Replay were gradually integrated， and a multi-scale convolutional neural network was built using techniques such as one-hot encoding to construct an improved DQN model. Finally， a visual simulation experiment was designed， and an ablation experiment was performed using evaluation indicators such as grid coverage. The experimental results show that the improved DQN model performs well in the coverage path planning task， and its performance is better than that of traditional DQN and other models. Compared with traditional DQN， its grid coverage rate is increased by 8.28%. Also， the grid repetition rate， path length， number of collisions with obstacles， and number of times exceeding boundaries are reduced by 28.44%， 45.14%， 77.31%， and 90.14%. The network convergence speed is significantly accelerated as well.

Key words: computer applications, deep reinforcement learning, coverage path planning, unmanned aerial vehicle, grid maps

中图分类号:

TP391

王兴旺,罗锦宽,张紫洋,迟玉璞,蒋金佑,于美铭. 基于改进型DQN的无人机覆盖路径规划算法[J]. 吉林大学学报(工学版), 2026, 56(9): 2444-2455.

Xing-wang WANG,Jin-kuan LUO,Zi-yang ZHANG,Yu-pu CHI,Jin-you JIANG,Mei-ming YU. Algorithm of coverage path planning for unmanned aerial vehicle based on improved DQN[J]. Journal of Jilin University(Engineering and Technology Edition), 2026, 56(9): 2444-2455.

图/表 14

图1

图2

图3

图4

图5

图6

图7

表1

表2

参数敏感性分析实验结果"

改进型DQN模型相关参数	栅格覆盖率/%	栅格重复率/%	路径长度	碰撞障碍物次数	超出边界次数
ω=1e-4、γ=0.99、 $G ? i, j = G ? (i', j')$ 时奖励值+0.25	97.90	35.83	154.41	5.46	0.34
ω=1e-3	98.37	31.33	144.91	3.97	0.51
ω=1e-5	83.89	71.03	291.75	13.11	1.97
γ=0.8	96.91	43.36	173.43	3.40	2.44
γ=0.9	97.85	39.97	164.51	4.41	1.54
$G ? i, j = G ? (i', j')$ 时奖励值+0	92.41	53.94	230.00	15.79	2.01
$G ? i, j = G ? (i', j')$ 时奖励值+0.5	98.04	35.67	158.67	9.03	0.27

表2

图8

图9

图10

表3

表4

参考文献 20

[1]	Buchelt A, Adrowitzer A, Kieseberg P, et al. Exploring artificial intelligence for applications of drones in forest ecology and management[J]. Forest Ecology and Management, 2024, 551: No.121530.
[2]	Bakirci M. Smart city air quality management through leveraging drones for precision monitoring[J]. Sustainable Cities and Society, 2024, 106: No.105390.
[3]	Ibrahim Z T, He J. Slam technology on disaster response[J]. World Journal of Engineering and Technology, 2024, 12(3): 695-714.
[4]	Xing B, Wang X, Yang L, et al. An algorithm of complete coverage path planning for unmanned surface vehicle based on reinforcement learning[J]. Journal of Marine Science and Engineering, 2023, 11(3): No.645.
[5]	Kim J. Autonomous robot vacuum system composed of a cleaner robot and a dust storage robot[J]. Journal of the Franklin Institute, 2024, 361(11): No.106938.
[6]	Wan S, Chen Z, Dong J. An efficiency-based interactive dynamic technique with interval-valued hesitant fuzzy constraint cone for rescue route planning[J]. Expert Systems with Applications, 2023, 231: No.120648.
[7]	Dijkstra E W. A note on two problems in connexion with graphs[J]. Numerische Mathematik, 1959, 1(1): 269-271.
[8]	Hart P E, Nilsson N J, Raphael B. A formal basis for the heuristic determination of minimum cost paths[J]. IEEE transactions on Systems Science and Cybernetics, 1968, 4(2): 100-107.
[9]	Xu R, Xia Y, Chen P. A Multi-Objective Particle Swarm Optimization Algorithm for Drone Path Planning in Forest Firefighting[C]∥The 11th International Conference on Electrical and Electronics Engineering(ICEEE), Marmaris,Turkiye,2024:523-527.
[10]	唐颂, 吴建源. 基于改进遗传算法的协同航迹规划方法[J]. 电光与控制, 2024, 31(7): 8-12, 26.
	Tang Song, Wu Jian-yuan. A Cooperative Trajectory Planning Method Based on Improved Genetic Algorithm[J]. Electronics Optics & Control, 2024, 31(7): 8-12, 26.
[11]	Luo J, Shun H, Hao Y. A path planning method for comprehensive area coverage utilizing neuron activity[C]∥The 5th International Conference on Robotics, Intelligent Control and Artificial Intelligence(RICAI),Hangzhou,China, 2023: 379-383.
[12]	屈博琛. 基于深度学习自动微分的无人机路径规划[J]. 自动化与仪表, 2025, 40(2): 66-72.
	Qu Bo-chen. Drone path planning based on deep learning with automatic differentiation[J]. Automation & Instrumentation, 2025, 40(2): 66-72.
[13]	Mnih V, Kavukcuoglu K, Silver D, et al. Playing atari with deep reinforcement learning[J/OL].[2025-02-16]. .
[14]	van Hasselt H, Guez A, Silver D. Deep reinforcement learning with double Q-learning[C]∥Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence, Phoenix, Arizona, 2016, 30(1): 2094-2100.
[15]	Wang Z, Schaul T, Hessel M, et al. Dueling network architectures for deep reinforcement learning[C]∥International Conference on Machine Learning, New York, USA, 2016: 1995-2003.
[16]	Schaul T, Quan J, Antonoglou I, et al. Prioritized experience replay[J/OL]. [2025-01-22]. .
[17]	Watkins C J C H. Learning from delayed rewards[D]. Cambridge: King's College, University of Cambridge,1989.
[18]	Jaderberg M, Simonyan K, Zisserman A. Spatial transformer networks[J]. Advances in Neural Information Processing Systems, 2015, 28: 2017-2025.
[19]	Hu J, Shen L, Sun G. Squeeze-and-excitation networks[C]∥Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, USA, 2018:7132-7141.
[20]	Brockman G, Cheung V, Pettersson L, et al. Openai gym[J/OL].[2025-01-26]. .

相关文章 15

[1]	巨涛,张文金,杨垚,火久元. 一种二阶决策的车联网边缘计算任务动态卸载方法[J]. 吉林大学学报(工学版), 2026, 56(7): 2006-2019.
[2]	张楠,张和茂,章涛,张艳军. 智能机器人自适应巡检路径栅格地图邻域搜索算法[J]. 吉林大学学报(工学版), 2026, 56(7): 2026-2033.
[3]	田成军,颜禹,崔仁伟,张晋通. 基于深度强化学习的机械臂自主抓取算法[J]. 吉林大学学报(工学版), 2026, 56(3): 662-669.
[4]	刘康琳,张泽宇,蒋婧雯,宫洵,陈垚. 基于分布鲁棒优化的无人机配送设施选址分配问题[J]. 吉林大学学报(工学版), 2026, 56(2): 464-472.
[5]	王琳虹,刘宇阳,刘子昱,鹿应佳,张宇恒,黄桂树. 基于YOLOv5的轻量化桥梁缺陷识别[J]. 吉林大学学报(工学版), 2025, 55(9): 2958-2968.
[6]	艾青林,刘元宵,杨佳豪. 基于MFF-STDC网络的室外复杂环境小目标语义分割方法[J]. 吉林大学学报(工学版), 2025, 55(8): 2681-2692.
[7]	张宇飞,王丽敏,赵建平,贾智尧,李明洋. 基于中心选择大逃杀优化算法的机器人逆运动学求解[J]. 吉林大学学报(工学版), 2025, 55(8): 2703-2710.
[8]	朴燕,康继元. RAUGAN：基于循环生成对抗网络的红外图像彩色化方法[J]. 吉林大学学报(工学版), 2025, 55(8): 2722-2731.
[9]	朱科,邢志明,康翔宇. 机械手多任务均衡策略[J]. 吉林大学学报(工学版), 2025, 55(8): 2782-2790.
[10]	车翔玖,李良. 融合全局与局部细粒度特征的图相似度度量算法[J]. 吉林大学学报(工学版), 2025, 55(7): 2365-2371.
[11]	李文辉,杨晨. 基于对比学习文本感知的小样本遥感图像分类[J]. 吉林大学学报(工学版), 2025, 55(7): 2393-2401.
[12]	庄珊娜,王君帅,白晶,杜京瑾,王正友. 基于三维卷积与自注意力机制的视频行人重识别[J]. 吉林大学学报(工学版), 2025, 55(7): 2409-2417.
[13]	刘琼昕,王甜甜,王亚男. 非支配排序粒子群遗传算法解决车辆位置路由问题[J]. 吉林大学学报(工学版), 2025, 55(7): 2464-2474.
[14]	王健,贾晨威. 面向智能网联车辆的轨迹预测模型[J]. 吉林大学学报(工学版), 2025, 55(6): 1963-1972.
[15]	车翔玖,孙雨鹏. 基于相似度随机游走聚合的图节点分类算法[J]. 吉林大学学报(工学版), 2025, 55(6): 2069-2075.

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed

参数	含义	值
ω	网络学习率	1e-4
γ	折扣因子	0.99
?	见式（9）	1e-5
μ	见式（9）	0.6
σ	见式（11）	0.4
H	二维栅格地图高度	10
W	二维栅格地图宽度	10
frame_max	最大训练帧数	1 000 000
frame_start	训练开始帧数	10 000
\|D\|	回放经验池最大样本容量	100 000
\|K\|	可选动作的总数	4
Batch_size	每次训练选取的样本数	32
epsilon_max	最大贪婪系数	1.0
epsilon_min	最小贪婪系数	0.01
epsilon_decay	至最小贪婪系数的帧数	30 000
max_steps	每回合最大步数	300
agent_num	智能体数量	1
update_freq	网络参数更新间隔/帧	1 000
obstacles_per	障碍物栅格的占比/%	16

DDQN	Dueing network	PER	栅格覆盖率/%	栅格重复率/%	路径长度	碰撞障碍物次数	超出边界次数	训练时间	#Param /M	#FLOPs /G
?	?	?	89.62	64.27	281.45	24.06	3.45	3：22：04	3.72	1.11
?	?	?	93.92	61.83	273.66	23.78	0.85	4：14：59	3.72	1.11
?	?	?	93.72	56.13	233.79	12.72	3.65	3：56：05	7.13	1.22
?	?	?	94.47	51.18	215.99	11.69	2.62	3：26：30	3.72	1.11
?	?	?	94.82	50.40	219.92	19.37	0.71	4：31：45	7.13	1.22
?	?	?	96.26	44.76	187.66	10.76	0.77	4：07：36	7.13	1.22
?	?	?	95.94	48.76	201.16	10.05	3.34	3：38：55	3.72	1.11
?	?	?	97.90	35.83	154.41	5.46	0.34	4：36：02	7.13	1.22

模型	H，W	max_steps	栅格覆盖率/%	栅格重复率/%	路径长度	碰撞障碍物次数	超出边界次数
DQN	10	300	89.62	64.27	281.45	24.06	3.45
DQN+①	10	300	93.92	61.83	273.66	23.78	0.85
DQN+②	10	300	93.72	56.13	233.79	12.72	3.65
DQN+③	10	300	94.47	51.18	215.99	11.69	2.62
改进型DQN	10	300	97.90	35.83	154.41	5.46	0.34
DQN	12	350	84.68	63.46	349.70	20.36	26.14
DQN+①	12	350	86.14	64.84	348.76	13.07	43.30
DQN+②	12	350	85.16	61.47	345.89	24.45	31.46
DQN+③	12	350	89.80	58.61	329.08	18.76	12.15
改进型DQN	12	350	94.13	57.00	320.95	13.00	5.46
DQN	15	400	80.77	53.10	399.98	29.49	19.75
DQN+①	15	400	84.64	50.86	400.00	30.84	30.76
DQN+②	15	400	83.14	51.68	399.99	30.51	35.20
DQN+③	15	400	87.60	52.56	399.94	18.17	25.78
改进型DQN	15	400	92.70	49.66	391.99	14.29	4.96