基于双向长短期记忆模型的起重机智能操控方法

doi:10.13229/j.cnki.jdxbgxb20190231

摘要/Abstract

摘要：

为解决地面指挥人员和司机协同操控起重机难度大的问题，提出一种基于双向长短期记忆（BiLSTM）模型的起重机智能操控方法，实现了对起重机的单人控制，降低了人力成本。首先，使用Kinect从指挥人员的一段动作指令中采集出人体关节点的坐标序列；然后，利用这些坐标构造人体关节向量，通过计算向量间的夹角和模比值构造出两种区分不同指令的特征矩阵；之后，将夹角特征矩阵输入到基于BiLSTM模型的指令识别网络，并与支持向量机（SVM）和反向传播（BP）神经网络的识别结果作比较；最后，将夹角和模比值特征矩阵进行融合识别，以进一步提升准确率。实验结果表明:本文指令识别网络具有较高的识别率；提出的融合识别方法有效地利用多种特征的信息，对训练集的识别准确率达99.13%，对测试集的识别准确率达96.75%。

关键词: 起重机智能操控, 吊运指令识别, 特征矩阵, 双向长短期记忆模型, 多特征融合识别

Abstract:

To solve the problem that it is difficult for commander on the ground and driver to operate cranes cooperatively, an intelligent control method of the crane based on BiLSTM model was proposed, which realizes single-person control of crane, thus, reducing manpower cost. Firstly, Kinect sensors were used to collect the coordinates of human joints from the commander's instructions. Then, the vectors of human joints were constructed using the coordinates, and two kinds of characteristic matrices that distinguish different instructions were constructed by calculating the angles and modulus ratios between the vectors. After that, the feature matrix of angle was input into the instruction recognition network based on BiLSTM model, and the recognition results were compared with those of support vector machine (SVM) and back propagation (BP) neural networks. Finally, the feature matrices of angle and modulus ratio were fused to improve the accuracy. The experimental results show that the instruction recognition network designed in this paper achieves the highest accuracy rate and the proposed fusion recognition method utilizes the information of multiple features effectively. For the training set, the recognition rate can reach 99.13%, and for the test set, this rate can reach 96.75%.

Key words: intelligent operation of crane, identification of lifting instructions, characteristic matrix, BiLSTM model, multi-features fusion recognition

中图分类号:

TH218

倪涛,刘海强,王林林,邹少元,张红彦,黄玲涛. 基于双向长短期记忆模型的起重机智能操控方法[J]. 吉林大学学报(工学版), 2020, 50(2): 445-453.

Tao NI,Hai-qiang LIU,Lin-lin WANG,Shao-yuan ZOU,Hong-yan ZHANG,Ling-tao HUANG. Intelligent manipulation method of crane based on BiLSTM model[J]. Journal of Jilin University(Engineering and Technology Edition), 2020, 50(2): 445-453.

图/表 18

图1

图2

图3

图4

表1

图5

图6

表2

模比值特征"

简称	模比值	简称	模比值
$r 1$	$r b, s l$	$r 7$	$r b, s r$
$r 2$	$r b, e l$	$r 8$	$r b, e r$
$r 3$	$r b, w l$	$r 9$	$r b, w r$
$r 4$	$r b, k l$	$r 10$	$r b, k r$
$r 5$	$r b, a l$	$r 11$	$r b, a r$
$r 6$	$r b, h$

表2

表3

向量角特征"

简称	夹角	简称	夹角
$θ 1$	$l s r, e r, l s s, s r$	$θ 9$	$l s r, e r, l s s, s r$
$θ 2$	$l b, s l, l s, s l$	$θ 10$	$l b, s r, l s, s r$
$θ 3$	$l s l, e l, l e l, w l$	$θ 11$	$l s r, e r, l e r, w r$
$θ 4$	$l b, e l, l s l, e l$	$θ 12$	$l b, e r, l s r, e r$
$θ 5$	$l b, h l, l h l, k l$	$θ 13$	$l b, h r, l h r, k r$
$θ 6$	$l h l, k l, l k l, a l$	$θ 14$	$l h r, k r, l k r, a r$
$θ 7$	$l b, k l, l h l, k l$	$θ 15$	$l b, k r, l h r, k r$
$θ 8$	$l b, s s, l s s, h$	$θ 16$	$l b s, s r, l s s, h$

表3

图7

图8

图9

图10

图11

表4

图12

表5

表6

参考文献 20

1	臧大进, 戚玉强. 塔式起重机智能监控系统的研制[J]. 冶金设备, 2009(1): 50-53.
	Zang Da-jin, Qi Yu-qiang. Development of intelligent monitoring system of tower cranes[J]. Metallurgical Equipment, 2009(1): 50-53.
2	潘斌, 汪小东. 塔式起重机安全管理的特点及对策[J]. 工业设计, 2011(6): 136.
	Pan Bin, Wang Xiao-dong. Safety management characteristics and countermeasures of tower crane[J]. Design Ideas, 2011(6): 136.
3	腾文花. 关于塔吊使用的安全监控措施[J]. 建筑安全, 2007, 22(11): 36-36.
	Teng Wen-hua. Safety monitoring measures for tower crane use[J]. Construction Safety, 2007, 22(11): 36-36.
4	李雄祥, 潘英俭, 廖拓. 起重机械的工作特点及发展趋势[J]. 中国水运: 理论版, 2007, 5(1): 171.
	Li Xiong-xiang, Pan Ying-jian, Liao Tuo. Working characteristics and development trend of hoisting machinery[J]. China Water Transport, 2007, 5(1): 171.
5	张敏. 起重机检验中危险因素的识别与控制[J]. 科技风, 2015(12): 100.
	Zhang Min. Identification and control of dangerous factors in crane inspection [J]. Technology Wind, 2015(12): 100.
6	尹浩. 港口门式起重机安全监控管理系统研究[D]. 武汉：武汉纺织大学机械工程与自动化学院, 2015.
	Yin Hao. Safety management characteristics and countermeasures of tower crane[D]. Wuhan： School of Mechanical Engineering and Automation, Wuhan Textile University, 2015.
7	贾秋枫. 大型履带式起重机吊装市场现状及发展趋势[J]. 石油化工建设, 2008, 30(5): 20-22.
	Jia Qiu-feng. On the large scale crawler cranes hoisting business in China[J]. Petroleum & Chemical Construction, 2008, 30(5): 20-22.
8	Zhang C C. Design of a crane intelligent control system[C]∥International Conference on Humanities and Social Science Research, Busan, South Korea, 2016: 861-864.
9	An J Q, Chen F, Chen X, et al. Three dimensional hoisting simulation system based on virtual reality for truck crane[C]∥2015 34th Chinese Control Conference (CCC), Hangzhou, China, 2015: 8892-8897.
10	祁家榕, 张昌伟. 行为识别技术的研究与发展[J]. 智能计算机与应用, 2017, 7(4): 24-26, 30.
	Qi Jia-rong, Zhang Chang-wei. Research and development of behavior recognition technology[J]. Intelligent Computer and Applications, 2017, 7(4): 24-26, 30.
11	倪涛, 赵泳嘉, 张红彦, 等. 基于Kinect动态手势识别的机械臂实时位姿控制系统[J]. 农业机械学报, 2017, 48(10): 417-423, 407.
	Ni Tao, Zhao Yong-jia, Zhang Hong-yan, et al. Real-time mechanical arm position and pose control system by dynamic hand gesture recognition based on kinect device[J]. Transactions of the Chinese Society for Agricultural Machinery, 2017, 48(10): 417-423, 407.
12	Canal G, Escalera S, Angulo C. A real-time human-robot interaction system based on gestures for assistive scenarios[J]. Computer Vision and Image Understanding, 2016, 149: 65-77.
13	张质文，王金诺，程文明, 等. 起重机设计手册[M]. 北京：中国铁道出版社, 2013.
14	石忠晓. 智能化起重机安全监控系统[D]. 大连：大连理工大学机械工程学院, 2003.
	Shi Zhong-xiao. Intelligent crane safety monitoring system[D]. Dalian： School of Mechanical Engineering, Dalian University of Technology, 2003.
15	韩旭. 应用Kinect的人体行为识别方法研究与系统设计[D]. 济南：山东大学控制科学与工程学院, 2013.
	Han Xu. The human behavior recognition research and system design using kinect[D]. Jinan： School of Control Science and Engineering, Shandong University, 2013.
16	Zhang S, Yang Y, Xiao J, et al. Fusing feometric features for skeleton-based action recognition using multilayer LSTM networks[J]. IEEE Transactions on Multimedia, 2018, 20(9): 2330-2343.
17	Cho K, Van M B, Gulcehre C, et al. Learning phrase representations using RNN encoder-decoder for statistical machine translation[C]∥Conference on Empirical Methods in Natural Language Processing (EMNLP), Doha, Qatar, 2014: 1724-1734.
18	王鑫, 吴际, 刘超, 等. 基于LSTM循环神经网络的故障时间序列预测[J]. 北京航空航天大学学报, 2018, 44(4): 772-784.
	Wang Xin, Wu Ji, Liu Chao, et al. Exploring LSTM based recurrent neural network for failure time series prediction[J]. Journal of Beijing University of Aeronautics and Astronautics, 2018, 44(4): 772-784.
19	Greff K, Srivastava R K, Koutník J, et al. LSTM: a search space odyssey[J]. IEEE Transactions on Neural Networks and Learning Systems, 2017, 28(10): 2222-2232.
20	GB5 082－1985起重吊运指挥信号[S].

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed

关节点	简称	关节点	简称
左手腕	wl	右手腕	wr
左肘	el	右肘	er
左肩	sl	右肩	sr
左脚踝	al	右脚踝	ar
左膝	kl	右膝	kr
左髋	hl	右髋	hr
肩椎	ss	基节点	b
头	h

识别模型	训练集/%	测试集/%
SVM	80.75	77.50
BP神经网络	89.13	85.25
图8网络	93.38	92.25

识别模型	训练集%	测试集/%
图8网络（输入特征矩阵Θ）	93.38	92.25
图8网络（输入特征矩阵R）	92.38	89.75
图9(a)网络（拼接R和Θ融合）	90.13	85.75
图9(b)网络（加权融合）	99.13	96.75

	预备	要主钩	要副钩	吊钩升	吊钩降	升臂	降臂	结束
预备	0.96	0	0	0.04	0	0	0	0
要主钩	0	0.94	0	0.06	0	0	0	0
要副钩	0	0	1.00	0	0	0	0	0
吊钩升	0.04	0.04	0	0.92	0	0	0	0
吊钩降	0.04	0	0	0	0.96	0	0	0
升臂	0	0	0	0	0	0.98	0.02	0
降臂	0	0	0	0	0	0.02	0.98	0
结束	0	0	0	0	0	0	0	1.00