基于深度强化学习的机械臂自主抓取算法

doi:10.13229/j.cnki.jdxbgxb.20240739

摘要/Abstract

摘要：

针对传统机械臂不具备自主学习能力，在未知环境中适应能力差等问题，提出了一种结合先验知识的深度强化学习算法，同时修改了奖励函数和经验池的设计，解决了训练前期数据质量低、收敛速度慢、学习效果不理想等问题，同时增强了其泛化性。仿真结果表明：改进后的模型相比于原始算法收敛速度提高了48.5%，成功率提升了8%，对比其他主流算法在收敛速度和成功率上都有明显提升。

关键词: 模式识别与智能系统, 深度强化学习, 先验知识, 机械臂

Abstract:

Aiming at the problems such as the lack of independent learning ability of traditional robotic arms and poor adaptability in unknown environments， this paper proposes a deep reinforcement learning algorithm combining prior knowledge， and modifies the design of reward function and experience pool to solve the problems such as low data quality， slow convergence speed and unsatisfactory learning effect in the early stage of training， while enhancing its generalization. The results on Pybullet simulation platform show that compared with the original algorithm， the convergence speed of the improved model is increased by 48.5%， and the success rate is increased by 8%. Compared with other mainstream algorithms， the convergence speed and success rate are significantly improved.

Key words: pattern recognition and intelligent system, deep reinforcement learning, priori knowledge, robot arm

中图分类号:

TP183

田成军,颜禹,崔仁伟,张晋通. 基于深度强化学习的机械臂自主抓取算法[J]. 吉林大学学报(工学版), 2026, 56(3): 662-669.

Cheng-jun TIAN,Yu YAN,Ren-wei CUI,Jin-tong ZHANG. Autonomous grasping algorithm of robotic arm based on deep reinforcement learning[J]. Journal of Jilin University(Engineering and Technology Edition), 2026, 56(3): 662-669.

图/表 12

图1

图2

图3

图4

图5

图6

图7

图8

图9

图10

图11

表1

参考文献 14

[1]	吕帅, 刘京. 基于深度强化学习的随机局部搜索启发式方法[J]. 吉林大学学报: 工学版, 2021, 51(4): 1420-1426.
	Shuai Lyu, Liu Jing. Stochastic local search heuristic method based on deep reinforcement learning[J]. Journal of Jilin University (Engineering and Technology Edition), 2021, 51(4): 1420-1426.
[2]	李保罡, 王宇, 孔凡伟, 等. 基于智能反射表面辅助和信息年龄度量的安全状态更新[J]. 吉林大学学报:工学版, 2023, 53(10): 3014-3025.
	Li Bao-gang, Wang Yu, Kong Fan-wei, et al. Security status updates based on intelligent reflecting surface assistance and age of information metrics[J]. Journal of Jilin University (Engineering and Technology Edition), 2023, 53(10): 3014-3025.
[3]	Schaul T, Quan J, Antonoglou I, et al. Prioritized experience replay[J/OL]. arXiv preprint arXiv:, 2015.
[4]	Rauber P, Ummadisingu A, Mutz F, et al. Hindsight policy gradients[J/OL]. arXiv preprint arXiv:, 2017.
[5]	Haarnoja T, Zhou A, Hartikainen K, et al. Soft actor-critic algorithms and applications[J/OL]. arXiv preprint arXiv:, 2018.
[6]	Rakelly K, Zhou A, Finn C, et al. Efficient off-policy meta-reinforcement learning via probabilistic context variables[C]∥International Conference on Machine Learning, PMLR, 2019: 5331-5340.
[7]	庄伟超, 丁昊楠, 董昊轩, 等. 信号交叉口网联电动汽车自适应学习生态驾驶策略[J]. 吉林大学学报: 工学版, 2023, 53(1): 82-93.
	Zhuang Wei-chao, Ding Hao-nan, Dong Hao-xuan, et al. Learning based eco⁃driving strategy of connected electric vehicle at signalized intersection[J]. Journal of Jilin University (Engineering and Technology Edition), 2023, 53(1): 82-93.
[8]	高敬鹏, 王国轩, 高路. 基于异步合作更新的LSTM-MADDPG多智能体协同决策算法[J]. 吉林大学学报: 工学版, 2024, 54(3): 797-806.
	Gao Jing-peng, Wang Guo-xuan, Gao Lu. LSTM⁃MADDPG multi⁃agent cooperative decision algorithm based on asynchronous collaborative update[J]. Journal of Jilin University (Engineering and Technology Edition), 2024, 54(3): 797-806.
[9]	张强, 文闻, 周晓东, 等. 基于改进TD3算法的机械臂智能规划方法研究[J]. 智能科学与技术学报, 2022, 4(2): 223-232.
	Zhang Qiang, Wen Wen, Zhou Xiao-dong, et al. Research on the manipulator intelligent trajectory planning method based on the improved TD3 algorithm[J]. Chinese Journal of Intelligent Science and Technology, 2022, 4(2): 223-232.
[10]	鲜斌, 张诗婧, 韩晓薇, 等. 基于强化学习的无人机吊挂负载系统轨迹规划[J]. 吉林大学学报:工学版, 2021, 51(6): 2259-2267.
	Xian Bin, Zhang Shi-jing, Han Xiao-wei, et al. Trajectory planning for unmanned aerial vehicle slung⁃payload aerial transportation system based on reinforcement learning[J]. Journal of Jilin University (Engineering and Technology Edition), 2021, 51(6): 2259-2267.
[11]	康朝海, 孙超, 荣垂霆, 等. 基于动态延迟策略更新的TD3 算法[J]. 吉林大学学报: 信息科学版, 2020, 38(4): 474-481.
	Kang Chao-hai, Sun Chao, Rong Chui-ting, et al. TD3 Algorithm with dynamic delayed policy update[J]. Journal of Jilin University (Information Science Edition), 2020, 38(4): 474-481.
[12]	刘庆强, 刘鹏云. 基于优先级经验回放的SAC强化学习算法[J]. 吉林大学学报: 信息科学版, 2021, 39(2): 192-199.
	Liu Qing-qiang, Liu Peng-yun. Soft actor critic reinforcement learning with prioritized experience replay[J]. Journal of Jilin University (Information Science Edition), 2021, 39(2): 192-199.
[13]	赵宏伟, 陈霄, 龙曼丽, 等. 基于改进PLSA分类器的目标分类算法[J]. 吉林大学学报: 工学版, 2012, 42(): 231-235.
	Zhao Hong-wei, Chen Xiao, Long Man-li, et al. Object classification algorithm based on improved PLSA[J]. Journal of Jilin University (Engineering and Technology Edition), 2012, 42(Sup.1): 231-235.
[14]	刘勇, 徐雷, 张楚晗. 面向文本游戏的深度强化学习模型[J]. 吉林大学学报: 工学版, 2022, 52(3): 666-674.
	Liu Yong, Xu Lei, Zhang Chu-han. Deep reinforcement learning model for text games[J]. Journal of Jilin University (Engineering and Technology Edition), 2022, 52(3): 666-674.

相关文章 15

[1]	朱科,邢志明,康翔宇. 机械手多任务均衡策略[J]. 吉林大学学报(工学版), 2025, 55(8): 2782-2790.
[2]	申自浩,高永生,王辉,刘沛骞,刘琨. 面向车联网隐私保护的深度确定性策略梯度缓存方法[J]. 吉林大学学报(工学版), 2025, 55(5): 1638-1647.
[3]	胡云峰,李佳敏,唐志国. 移动装弹机械臂的逆运动学多种群灰狼算法求解方法[J]. 吉林大学学报(工学版), 2025, 55(4): 1443-1452.
[4]	赵孟雪,车翔玖,徐欢,刘全乐. 基于先验知识优化的医学图像候选区域生成方法[J]. 吉林大学学报(工学版), 2025, 55(2): 722-730.
[5]	邢宏军,徐宇哲,周忠歌,刘彦清,丁亮,陈金宝. 考虑任务优先级的冗余轮式移动机械臂位形优化[J]. 吉林大学学报(工学版), 2025, 55(11): 3507-3520.
[6]	胡伟超,杨镇铭,于鹏程,陈艳艳,马社强. 基于深度强化学习的自动驾驶车辆与行人交互建模[J]. 吉林大学学报(工学版), 2025, 55(10): 3180-3188.
[7]	沈鹏,李小华,刘辉. 具有输入缓冲的机械臂预设定有限时间跟踪控制[J]. 吉林大学学报(工学版), 2025, 55(1): 63-73.
[8]	左从菊,秦国栋,潘洪涛,程勇,周浦城,秦晓燕,汪卫华. 聚变堆大型机械臂鲁棒自适应精度控制[J]. 吉林大学学报(工学版), 2025, 55(1): 52-62.
[9]	朱圣杰,王宣,徐芳,彭佳琦,王远超. 机载广域遥感图像的尺度归一化目标检测方法[J]. 吉林大学学报(工学版), 2024, 54(8): 2329-2337.
[10]	汤小红,龚永健. 考虑能耗与负载因素的抓取机械臂运动轨迹规划算法[J]. 吉林大学学报(工学版), 2024, 54(7): 1862-1868.
[11]	朱广贺,朱智强,袁逸萍. 连续生产流水线深度强化学习优化调度算法[J]. 吉林大学学报(工学版), 2024, 54(7): 2086-2092.
[12]	王宏志,王婷婷,兰淼淼,徐硕. 基于位置跟踪的机械臂多电机新型滑模控制策略[J]. 吉林大学学报(工学版), 2024, 54(5): 1443-1458.
[13]	高敬鹏,王国轩,高路. 基于异步合作更新的LSTM-MADDPG多智能体协同决策算法[J]. 吉林大学学报(工学版), 2024, 54(3): 797-806.
[14]	郑玉坤,孙如月,李凤鸣,刘义祥,李东广,宋锐. 液压机械臂集中式驱控系统研究与应用[J]. 吉林大学学报(工学版), 2024, 54(11): 3358-3371.
[15]	张健,李青扬,李丹,姜夏,雷艳红,季亚平. 基于深度强化学习的自动驾驶车辆专用道汇入引导[J]. 吉林大学学报(工学版), 2023, 53(9): 2508-2518.

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed

实验物体	每100次成功次数	识别IOU
螺丝刀	92	0.90
订书器	99	0.94
香蕉	80	0.88
锤子	88	0.88
牙膏	96	0.92
柠檬	82	0.80
网球	92	0.82