基于姿态估计的物体3D运动参数测量方法

doi:10.13229/j.cnki.jdxbgxb.20210981

摘要/Abstract

摘要：

针对传统的物体3D运动参数测量方法存在测量成本高昂、数据融合复杂、提取特征少等问题，提出了一种基于姿态估计的物体3D运动参数非接触式测量方法。首先，使用高帧率同步采集设备从不同视角捕获物体运动视频，弥补了单视角捕获目标容易出现盲区的缺点。其次，使用基于深度迁移学习的2D姿态估计框架DeepLabCut训练目标的2D姿态估计模型，利用模型估计物体不同视角的2D姿态，弥补了传统视觉方法提取特征少的缺点。再次，基于三角测量原理融合不同方向物体的2D姿态估计为3D姿态估计。最后，根据测量需要，利用3D姿态估计的结果可以计算物体指定关键点的位移、速度、姿态角、频率等3D运动参数。以行星摆和人手的运动测量为例，测量了二者的3D运动参数并分析了误差来源以及本研究的优缺点。实验结果表明，系统测量准确，可视化结果正确反映了实验对象的运动情况。该方法为非接触式测量物体3D运动参数提供了新的思路。

关键词: 计算机应用, 运动测量, 深度迁移学习, 姿态估计, 三角测量

Abstract:

Aiming at the problems of high measurement cost， complex data fusion， and few extracted features in traditional 3D motion parameter measurement methods， a non-contact measurement method of 3D motion parameters based on pose estimation is proposed. First of all， the use of high frame rate synchronous acquisition equipment to capture object motion video from different perspectives makes up for the shortcomings of blind spots that are prone to single-view capture targets. Secondly， using the 2D pose estimation model of the training target based on the deep transfer learning 2D pose estimation framework DeepLabCut， and using the model to estimate the 2D pose of the object from different perspectives， makes up for the shortcomings of the traditional vision method with fewer features. Third， the 2D pose estimation of objects in different directions is merged into a 3D pose estimation based on the principle of triangulation. Finally， according to the measurement requirements， the 3D motion parameters such as the displacement， speed， pose angle， frequency of the specified key points of the object can be calculated using the results of the 3D pose estimation. Taking the motion measurement of the planetary pendulum and the human hand as an example， we measured the 3D motion parameters of the two and analyzed the sources of error and the advantages and disadvantages of this paper. The experimental results show that the system measurement is accurate， and the visualization results correctly reflect the movement of the experimental subjects. This method provides a new idea for the non-contact measurement of 3D motion parameters of objects.

Key words: computer application, motion measurement, deep transfer learning, pose estimation, triangulation

中图分类号:

TP183

王连明,吴鑫. 基于姿态估计的物体3D运动参数测量方法[J]. 吉林大学学报(工学版), 2023, 53(7): 2099-2108.

Lian-ming WANG,Xin WU. Method for 3D motion parameter measurement based on pose estimation[J]. Journal of Jilin University(Engineering and Technology Edition), 2023, 53(7): 2099-2108.

图/表 12

图1

图2

表1

设备参数"

相机型号	OSG130?210UC
像素尺寸	$4.8 ? μ m × 4.8 ? μ m$
颜色空间	RGB
最高分辨率	1280×1024
初始帧率	210 帧/s
触发方式	软触发或者硬触发
数据接口	USB3.0
计算机型号	Dell
CPU	Inter@Xeon SP@6248R@3GHz
RAM	128.0 GB
GPU	RTX3090
系统类型	64?bit 操作系统 X64?based
操作系统	Windows 10

表1

图3

图4

图5

图6

图7

图8

图9

图10

图11

参考文献 22

1	Haydari A, Yilmaz Y. Deep reinforcement learning for intelligent transportation systems: a survey[J]. IEEE Transactions on Intelligent Transportation Systems, 2020.
2	Ju F, Zhu J, Shao B, et al. CopulaNet: learning residue co-evolution directly from multiple sequence alignment for protein structure prediction[J]. Nature Communications, 2021, 12(1): 1-9.
3	Hermann J, Schätzle Z, Noé F. Deep-neural-network solution of the electronic Schrödinger equation[J]. Nature Chemistry, 2020, 12(10): 891-897.
4	Wang T M, Tao Y, Liu H. Current researches and future development trend of intelligent robot: a review[J]. International Journal of Automation and Computing, 2018, 15(5): 525-546.
5	Zhao H, Wang Z. Motion measurement using inertial sensors, ultrasonic sensors, and magnetometers with extended Kalman filter for data fusion[J]. IEEE Sensors Journal, 2011, 12(5): 943-953.
6	Zeng H, Zhao Y. Sensing movement: Microsensors for body motion measurement[J]. Sensors, 2011, 11(1): 638-660.
7	Meinhart C D, Wereley S T, Santiago J G. PIV measurements of a microchannel flow[J]. Experiments in Fluids, 1999, 27(5): 414-419.
8	Sabel J C. Optical 3D motion measurement[C]∥IEEE Instrumentation and Measurement Technology Conference and IMEKO Tec, Brussels, Belgium, 1996: 367-370.
9	Eltanany A S, Elwan M S, Amein A S. Key Point Detection Techniques[M]. Cham: Springer International Publishing, 2020: 901-911.
10	Shapiro L, Stockman G. Computer Vision[M]. Upper Saddle River: Prentice Hall, 2001.
11	Needham L, Evans M, Cosker D P. The accuracy of several pose estimation methods for 3D joint centre localisation[J]. Scientific Reports, 11(1):No.20673.
12	Nath T, Mathis A, Chen A C, et al. Using DeepLabCut for 3D markerless pose estimation across species and behaviors[J]. Nature Protocols, 2019, 14(7): 2152-2176.
13	He K, Zhang X, Ren S, et al. Deep residual learning for image recognition[C]∥Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, USA, 2016: 770-778.
14	Howard A, Sandler M, Chu G, et al. Searching for mobilenetv3[C]∥Proceedings of the IEEE/CVF International Conference on Computer Vision, Seoul, Korea, 2019: 1314-1324.
15	Tan M, Le Q. Efficientnet: rethinking model scaling for convolutional neural networks[C]∥International Conference on Machine Learning, New York, USA 2019: 6105-6114.
16	Mathis A, Biasi T, Schneider S, et al. Pretraining boosts out-of-domain robustness for pose estimation[C]∥Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, Online, 2021: 1859-1868.
17	Szeliski R. Computer Vision: Algorithms and Applications[M]. Berlin: Springer Science & Business Media, 2010.
18	Zhang Z. A flexible new technique for camera calibration[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2000, 22(11): 1330-1334.
19	Karashchuk P, Rupp K L, Dickinson E S, et al. Anipose: a toolkit for robust markerless 3D pose estimation[J]. Cell Reports, 2021, 36(13):No.109730.
20	Xu Z, Chang X, Xu F, et al. L 1 / 2 regularization: A thresholding representation theory and a fast solver[J]. IEEE Transactions on Neural Networks and Learning Systems, 2012, 23(7): 1013-1027.
21	Triggs B, McLauchlan P F, Hartley R I, et al. Bundle adjustment—a modern synthesis[C]//International Workshop on Vision Algorithms, Berlin, Germany, 1999: 298-372.
22	Roithmayr C M, Hodges D H. Dynamics: theory and application of Kane's method[J]. Journal of Computational and Nonlinear Dynamics, 2016, 11(6):No. 066501.

相关文章 15

[1]	张振海,季坤,党建武. 基于桥梁裂缝识别模型的桥梁裂缝病害识别方法[J]. 吉林大学学报(工学版), 2023, 53(5): 1418-1426.
[2]	刘培勇,董洁,谢罗峰,朱杨洋,殷国富. 基于多支路卷积神经网络的磁瓦表面缺陷检测算法[J]. 吉林大学学报(工学版), 2023, 53(5): 1449-1457.
[3]	张则强,梁巍,谢梦柯,郑红斌. 混流双边拆卸线平衡问题的精英差分进化算法[J]. 吉林大学学报(工学版), 2023, 53(5): 1297-1304.
[4]	于鹏,朴燕. 基于多尺度特征的行人重识别属性提取新方法[J]. 吉林大学学报(工学版), 2023, 53(4): 1155-1162.
[5]	姜宇,潘家铮,陈何淮,符凌智,齐红. 基于分割方法的繁体中文报纸文本检测[J]. 吉林大学学报(工学版), 2023, 53(4): 1146-1154.
[6]	吴振宇,刘小飞,王义普. 基于DKRRT^*-APF算法的无人系统轨迹规划[J]. 吉林大学学报(工学版), 2023, 53(3): 781-791.
[7]	陶博,颜伏伍,尹智帅,武冬梅. 基于高精度地图增强的三维目标检测算法[J]. 吉林大学学报(工学版), 2023, 53(3): 802-809.
[8]	薛珊,张亚亮,吕琼莹,曹国华. 复杂背景下的反无人机系统目标检测算法[J]. 吉林大学学报(工学版), 2023, 53(3): 891-901.
[9]	潘弘洋,刘昭,杨波,孙庚,刘衍珩. 基于新一代通信技术的无人机系统群体智能方法综述[J]. 吉林大学学报(工学版), 2023, 53(3): 629-642.
[10]	何颖,樊俊松,王巍,孙庚,刘衍珩. 无人机空地安全通信与航迹规划的多目标联合优化方法[J]. 吉林大学学报(工学版), 2023, 53(3): 913-922.
[11]	郭鹏,赵文超,雷坤. 基于改进Jaya算法的双资源约束柔性作业车间调度[J]. 吉林大学学报(工学版), 2023, 53(2): 480-487.
[12]	刘近贞,高国辉,熊慧. 用于脑组织分割的多尺度注意网络[J]. 吉林大学学报(工学版), 2023, 53(2): 576-583.
[13]	祁贤雨,王巍,王琳,赵玉飞,董彦鹏. 基于物体语义栅格地图的语义拓扑地图构建方法[J]. 吉林大学学报(工学版), 2023, 53(2): 569-575.
[14]	时小虎,吴佳琦,吴春国,程石,翁小辉,常志勇. 基于残差网络的弯道增强车道线检测方法[J]. 吉林大学学报(工学版), 2023, 53(2): 584-592.
[15]	曲福恒,丁天雨,陆洋,杨勇,胡雅婷. 基于邻域相似性的图像码字快速搜索算法[J]. 吉林大学学报(工学版), 2022, 52(8): 1865-1871.

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed