Journal of Jilin University(Engineering and Technology Edition) ›› 2024, Vol. 54 ›› Issue (12): 3423-3432.doi: 10.13229/j.cnki.jdxbgxb.20230087

    Next Articles

Target grasping network technology of robot manipulator based on attention mechanism

Bin ZHAO1,2,3(),Cheng-dong WU1,3,Xue-jiao ZHANG3,Ruo-huai SUN1,Yang JIANG3   

  1. 1.College of Information Science and Engineering Northeastern University,Shenyang 110819,China
    2.SIASUN Robot & Automation Co. ,Ltd. ,Shenyang 110168,China
    3.Faculty of Robot Science and Engineering Northeastern University,Shenyang 110169,China
  • Received:2023-01-31 Online:2024-12-01 Published:2025-01-24

Abstract:

In order to solve the problem of single-object grasping detection of robotic arms, a SqueezeNet model algorithm based on attention mechanism (CBAM) was proposed. Firstly, the deep vision grasping system was described, and the hand-eye calibration of the grasping system was completed. The data set was preprocessed by randomly clipping, flipping, adjusting contrast, and increasing noise, effectively expanding the object capture data set. Secondly, a lightweight SqueezeNet model was introduced. It uses a five-parameter method to characterize the two-dimensional grab frame, which can complete the target capture without increasing the difficulty of network design. Thirdly, a plug-and-play network with an attention mechanism was introduced to weight the incoming feature maps in the channel and spatial dimensions. The SqueezeNet model-grabbing network was optimized and improved. Finally, the improved CBAM-SqueezeNet algorithm was verified on the public data sets Cornell grasping dataset and Jacquard dataset. The grab detection accuracy is 94.8% and 96.4%, accuracy increased 2% than the SqueezeNet network. The CBAM-SqueezeNet network grabbing method has a reasoning speed of 15 ms, which balances grabbing accuracy and running speed. The paper conducted experiments on the Kinova and SIASUN arm, and the success rate of network capture was 93%, which was faster and more efficient.

Key words: grab detection, attention mechanism, squeezenet, single-target object detection, deep learning

CLC Number: 

  • TP242.6

Fig.1

Kinova grasping system"

Fig.2

Depth camera calibration with Intel RealSense D435"

Fig.3

Schematic diagram of eye-on-hand calibration"

Fig.4

5D grasp parameter"

Fig.5

Channel attention and spatial attention"

Fig.6

Overall structure of the CBAM-SqueezeNet network"

Fig.7

Schematic diagram of the grasping system flow"

Table 1

Description of the public Grasping Datasets"

数据集类型物体个数RGB-D图像抓取个数
CornellRGB-D24010358019
JacquardRGB-D11 00054 000110 000

Fig.8

Selected detection outputs of the proposed model on different dataset"

Table 2

Parameters of Squeeze and CBAM- SqueezeNet"

网络类型总参数参数大小/MB
Squeeze209 1760.80
CBAM-Squeeze226 3320.86

Table 3

Detection performance of the proposed model on Cornell dataset"

网络类型准确率/%帧/s交并比
Squeeze92.6970.79
CBAM-Squeeze94.8670.81

Table 4

Detection performance of the proposed model on the Jacquard dataset"

网络类型准确率/%帧/s交并比
Squeeze93.2970.81
CBAM-Squeeze96.4670.84

Fig.9

LOSS and accuracy of Capture detection based on attention mechanism"

Fig.10

SIASUN UR grasping experiment under singleobject scene"

1 Zhang H B, Zhou X W, Lan X G, et al. A real-time robotic grasping approach with oriented anchor box[J]. IEEE Transactions on Systems, Man, and Cybernetics Systems, 2021, 50(5): 3014-3025.
2 Patten T, Park K, Vincze M. DGCM-Net: dense geometrical correspondence matching network for incremental experience-based robotic grasping[J/OL]. [2023-01-20].
3 Valarezo Añazco E, Rivera Lopez P, Park N, et al. Natural object manipulation using anthropomorphic robotic hand through deep reinforcement learning and deep grasping probability network[J]. Applied Intelligence, 2021,51(2): 1041-1055.
4 Lan R, Sun L, Liu Z, et al. MADNet: a fast and lightweight network for single-image super resolution[J]. IEEE Transactions on Cybernetics, 2021,51(3): 1443-1453.
5 Xiao Z J, Yang X D, Wei X, et al. Improved lightweight network in image recognition[J]. Jisuanji Kexue Yu Tansuo, 2021, 15(4): 743-753.
6 Li M, He Z, Zhu Y, et al. A method of grasping detection for kiwifruit harvesting robot based on deep learning[J]. Agronomy, 2022, 12(12): No.3096.
7 Li H, Cheng H. Lightweight neural network design for complex verification code recognition task[J]. Computer Systems and Applications, 2021, 4: No.247.
8 Song T N, Qin W W, Liang Z, et al. Improved dual-channel attention mechanism image classification method for lightweight network[J]. Hangkong Bingqi, 2021, 28(5): 81-85.
9 Zhao B, Wu C D, Zou F S, et al. Research on small sample multi-target grasping technology based on transfer learning[J]. Sensors, 2023, 23(13): No.5826.
10 Kim D, Jo H, Song J. Irregular depth tiles: automatically generated data used for network-based robotic grasping in 2D dense clutter[J]. International Journal of Control, Automation, and Systems, 2021, 19(10): 3428-3434.
11 Kumra S, Joshi S, Sahin F. GR-ConvNet v2: a real-time multi-grasp detection network for robotic grasping[J]. Sensors, 2022, 22(16): No.6208.
12 Wang D, Liu C, Chang F, et al. High-performance pixel-level grasp detection based on adaptive grasping and grasp-aware network[J].IEEE Transactions on Industrial Electronics, 2022,69(11): 11611-11621.
13 Himeur C, Lejemble T, Pellegrini T, et al. PCEDNet: a lightweight neural network for fast and interactive edge detection in 3D point clouds[J]. ACM Transactions on Graphics, 2022, 41(1):No. 3481804.
[1] Lei ZHANG,Jing JIAO,Bo-xin LI,Yan-jie ZHOU. Large capacity semi structured data extraction algorithm combining machine learning and deep learning [J]. Journal of Jilin University(Engineering and Technology Edition), 2024, 54(9): 2631-2637.
[2] Lu Li,Jun-qi Song,Ming Zhu,He-qun Tan,Yu-fan Zhou,Chao-qi Sun,Cheng-yu Zhou. Object extraction of yellow catfish based on RGHS image enhancement and improved YOLOv5 network [J]. Journal of Jilin University(Engineering and Technology Edition), 2024, 54(9): 2638-2645.
[3] Ping YU,Kang ZHAO,Jie CAO. Rolling bearing fault diagnosis based on optimized A-BiLSTM [J]. Journal of Jilin University(Engineering and Technology Edition), 2024, 54(8): 2156-2166.
[4] Yun-zuo ZHANG,Yu-xin ZHENG,Cun-yu WU,Tian ZHANG. Accurate lane detection of complex environment based on double feature extraction network [J]. Journal of Jilin University(Engineering and Technology Edition), 2024, 54(7): 1894-1902.
[5] Bai-you QIAO,Tong WU,Lu YANG,You-wen JIANG. A text sentiment analysis method based on BiGRU and capsule network [J]. Journal of Jilin University(Engineering and Technology Edition), 2024, 54(7): 2026-2037.
[6] Xin-gang GUO,Ying-chen HE,Chao CHENG. Noise-resistant multistep image super resolution network [J]. Journal of Jilin University(Engineering and Technology Edition), 2024, 54(7): 2063-2071.
[7] Li-ping ZHANG,Bin-yu LIU,Song LI,Zhong-xiao HAO. Trajectory k nearest neighbor query method based on sparse multi-head attention [J]. Journal of Jilin University(Engineering and Technology Edition), 2024, 54(6): 1756-1766.
[8] Ming-hui SUN,Hao XUE,Yu-bo JIN,Wei-dong QU,Gui-he QIN. Video saliency prediction with collective spatio-temporal attention [J]. Journal of Jilin University(Engineering and Technology Edition), 2024, 54(6): 1767-1776.
[9] Yu-kai LU,Shuai-ke YUAN,Shu-sheng XIONG,Shao-peng ZHU,Ning ZHANG. High precision detection system for automotive paint defects [J]. Journal of Jilin University(Engineering and Technology Edition), 2024, 54(5): 1205-1213.
[10] Yun-long GAO,Ming REN,Chuan WU,Wen GAO. An improved anchor-free model based on attention mechanism for ship detection [J]. Journal of Jilin University(Engineering and Technology Edition), 2024, 54(5): 1407-1416.
[11] Xiong-fei LI,Zi-xuan SONG,Rui ZHU,Xiao-li ZHANG. Remote sensing change detection model based on multi⁃scale fusion [J]. Journal of Jilin University(Engineering and Technology Edition), 2024, 54(2): 516-523.
[12] Guo-jun YANG,Ya-hui QI,Xiu-ming SHI. Review of bridge crack detection based on digital image technology [J]. Journal of Jilin University(Engineering and Technology Edition), 2024, 54(2): 313-332.
[13] Xiao-xu LI,Wen-juan AN,Ji-jie WU,Zhen LI,Ke ZHANG,Zhan-yu MA. Channel attention bilinear metric network [J]. Journal of Jilin University(Engineering and Technology Edition), 2024, 54(2): 524-532.
[14] Yong WANG,Yu-xiao BIAN,Xin-chao LI,Chun-ming XU,Gang PENG,Ji-kui WANG. Image dehazing algorithm based on multiscale encoding decoding neural network [J]. Journal of Jilin University(Engineering and Technology Edition), 2024, 54(12): 3626-3636.
[15] Na CHE,Yi-ming ZHU,Jian ZHAO,Lei SUN,Li-juan SHI,Xian-wei ZENG. Connectionism based audio-visual speech recognition method [J]. Journal of Jilin University(Engineering and Technology Edition), 2024, 54(10): 2984-2993.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
[1] LI Shoutao, LI Yuanchun. Autonomous Mobile Robot Control Algorithm Based on Hierarchical Fuzzy Behaviors in Unknown Environments[J]. 吉林大学学报(工学版), 2005, 35(04): 391 -397 .
[2] Liu Qing-min,Wang Long-shan,Chen Xiang-wei,Li Guo-fa. Ball nut detection by machine vision[J]. 吉林大学学报(工学版), 2006, 36(04): 534 -538 .
[3] Li Hong-ying; Shi Wei-guang;Gan Shu-cai. Electromagnetic properties and microwave absorbing property
of Z type hexaferrite Ba3-xLaxCo2Fe24O41
[J]. 吉林大学学报(工学版), 2006, 36(06): 856 -0860 .
[4] Zhang Quan-fa,Li Ming-zhe,Sun Gang,Ge Xin . Comparison between flexible and rigid blank-holding in multi-point forming[J]. 吉林大学学报(工学版), 2007, 37(01): 25 -30 .
[5] . [J]. 吉林大学学报(工学版), 2007, 37(04): 0 .
[6] Li Yue-ying,Liu Yong-bing,Chen Hua . Surface hardening and tribological properties of a cam materials[J]. 吉林大学学报(工学版), 2007, 37(05): 1064 -1068 .
[7] Feng Hao,Xi Jian-feng,Jiao Cheng-wu . Placement of roadside traffic signs based on visibility distance[J]. 吉林大学学报(工学版), 2007, 37(04): 782 -785 .
[8] Zhang He-sheng, Zhang Yi, Wen Hui-min, Hu Dong-cheng . Estimation approaches of average link travel time using GPS data[J]. 吉林大学学报(工学版), 2007, 37(03): 533 -0537 .
[9] Yang Shu-kai, Song Chuan-xue, An Xiao-juan, Cai Zhang-lin . Analyzing effects of suspension bushing elasticity
on vehicle yaw response character with virtual prototype method
[J]. 吉林大学学报(工学版), 2007, 37(05): 994 -0999 .
[10] . [J]. 吉林大学学报(工学版), 2007, 37(06): 1284 -1287 .