Journal of Jilin University(Engineering and Technology Edition) ›› 2023, Vol. 53 ›› Issue (7): 2099-2108.doi: 10.13229/j.cnki.jdxbgxb.20210981

Previous Articles    

Method for 3D motion parameter measurement based on pose estimation

Lian-ming WANG1(),Xin WU2   

  1. 1.School of Marine Information Engineering，Hainan Tropical Ocean University，Sanya 572022，China
    2.School of Physics，Northeast Normal University，Changchun 130024，China
  • Received:2021-09-28 Online:2023-07-01 Published:2023-07-20

RICH HTML

 7   

PDF (PC)

247

Abstract:

Aiming at the problems of high measurement cost， complex data fusion， and few extracted features in traditional 3D motion parameter measurement methods， a non-contact measurement method of 3D motion parameters based on pose estimation is proposed. First of all， the use of high frame rate synchronous acquisition equipment to capture object motion video from different perspectives makes up for the shortcomings of blind spots that are prone to single-view capture targets. Secondly， using the 2D pose estimation model of the training target based on the deep transfer learning 2D pose estimation framework DeepLabCut， and using the model to estimate the 2D pose of the object from different perspectives， makes up for the shortcomings of the traditional vision method with fewer features. Third， the 2D pose estimation of objects in different directions is merged into a 3D pose estimation based on the principle of triangulation. Finally， according to the measurement requirements， the 3D motion parameters such as the displacement， speed， pose angle， frequency of the specified key points of the object can be calculated using the results of the 3D pose estimation. Taking the motion measurement of the planetary pendulum and the human hand as an example， we measured the 3D motion parameters of the two and analyzed the sources of error and the advantages and disadvantages of this paper. The experimental results show that the system measurement is accurate， and the visualization results correctly reflect the movement of the experimental subjects. This method provides a new idea for the non-contact measurement of 3D motion parameters of objects.

Key words: computer application, motion measurement, deep transfer learning, pose estimation, triangulation

CLC Number: 

  • TP183

Fig.1

Structure of measurement system and data processing flow"

Fig.2

Illustration of three pose angles determinedby any three key points"

Table 1

Device parameters"

相机型号OSG130?210UC
像素尺寸4.8?μm×4.8?μm
颜色空间RGB
最高分辨率1280×1024
初始帧率210 帧/s
触发方式软触发或者硬触发
数据接口USB3.0
计算机型号Dell
CPUInter@Xeon SP@6248R@3GHz
RAM128.0 GB
GPURTX3090
系统类型64?bit 操作系统 X64?based
操作系统Windows 10

Fig.3

Examples of planetary pendulum labeling and3D pose estimation"

Fig.4

Visualization of 3D motion trajectory of a key point of planetary pendulum"

Fig.5

Visualization of 2D motion trajectory of a keypoint of planetary pendulum"

Fig.6

Visualization of motion parameters of a keypoint of planetary pendulum"

Fig.7

Examples of hand annotation and 3D poseestimation"

Fig.8

Visualization of 3D movement trajectory of second key point of middle finger （countedfrom end of fingertip）"

Fig.9

Visualization of 2D movement trajectory ofsecond key point of middle finger （counted from end of fingertip）"

Fig.10

Visualization of movement parameters of second key point of middle finger（counted from end of fingertip）"

Fig.11

Change of middle finger angle"

1 Haydari A, Yilmaz Y. Deep reinforcement learning for intelligent transportation systems: a survey[J]. IEEE Transactions on Intelligent Transportation Systems, 2020.
2 Ju F, Zhu J, Shao B, et al. CopulaNet: learning residue co-evolution directly from multiple sequence alignment for protein structure prediction[J]. Nature Communications, 2021, 12(1): 1-9.
3 Hermann J, Schätzle Z, Noé F. Deep-neural-network solution of the electronic Schrödinger equation[J]. Nature Chemistry, 2020, 12(10): 891-897.
4 Wang T M, Tao Y, Liu H. Current researches and future development trend of intelligent robot: a review[J]. International Journal of Automation and Computing, 2018, 15(5): 525-546.
5 Zhao H, Wang Z. Motion measurement using inertial sensors, ultrasonic sensors, and magnetometers with extended Kalman filter for data fusion[J]. IEEE Sensors Journal, 2011, 12(5): 943-953.
6 Zeng H, Zhao Y. Sensing movement: Microsensors for body motion measurement[J]. Sensors, 2011, 11(1): 638-660.
7 Meinhart C D, Wereley S T, Santiago J G. PIV measurements of a microchannel flow[J]. Experiments in Fluids, 1999, 27(5): 414-419.
8 Sabel J C. Optical 3D motion measurement[C]∥IEEE Instrumentation and Measurement Technology Conference and IMEKO Tec, Brussels, Belgium, 1996: 367-370.
9 Eltanany A S, Elwan M S, Amein A S. Key Point Detection Techniques[M]. Cham: Springer International Publishing, 2020: 901-911.
10 Shapiro L, Stockman G. Computer Vision[M]. Upper Saddle River: Prentice Hall, 2001.
11 Needham L, Evans M, Cosker D P. The accuracy of several pose estimation methods for 3D joint centre localisation[J]. Scientific Reports, 11(1):No.20673.
12 Nath T, Mathis A, Chen A C, et al. Using DeepLabCut for 3D markerless pose estimation across species and behaviors[J]. Nature Protocols, 2019, 14(7): 2152-2176.
13 He K, Zhang X, Ren S, et al. Deep residual learning for image recognition[C]∥Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, USA, 2016: 770-778.
14 Howard A, Sandler M, Chu G, et al. Searching for mobilenetv3[C]∥Proceedings of the IEEE/CVF International Conference on Computer Vision, Seoul, Korea, 2019: 1314-1324.
15 Tan M, Le Q. Efficientnet: rethinking model scaling for convolutional neural networks[C]∥International Conference on Machine Learning, New York, USA 2019: 6105-6114.
16 Mathis A, Biasi T, Schneider S, et al. Pretraining boosts out-of-domain robustness for pose estimation[C]∥Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, Online, 2021: 1859-1868.
17 Szeliski R. Computer Vision: Algorithms and Applications[M]. Berlin: Springer Science & Business Media, 2010.
18 Zhang Z. A flexible new technique for camera calibration[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2000, 22(11): 1330-1334.
19 Karashchuk P, Rupp K L, Dickinson E S, et al. Anipose: a toolkit for robust markerless 3D pose estimation[J]. Cell Reports, 2021, 36(13):No.109730.
20 Xu Z, Chang X, Xu F, et al. L 1 / 2 regularization: A thresholding representation theory and a fast solver[J]. IEEE Transactions on Neural Networks and Learning Systems, 2012, 23(7): 1013-1027.
21 Triggs B, McLauchlan P F, Hartley R I, et al. Bundle adjustment—a modern synthesis[C]//International Workshop on Vision Algorithms, Berlin, Germany, 1999: 298-372.
22 Roithmayr C M, Hodges D H. Dynamics: theory and application of Kane's method[J]. Journal of Computational and Nonlinear Dynamics, 2016, 11(6):No. 066501.
[1] Zhen-hai ZHANG,Kun JI,Jian-wu DANG. Crack identification method for bridge based on BCEM model [J]. Journal of Jilin University(Engineering and Technology Edition), 2023, 53(5): 1418-1426.
[2] Pei-yong LIU,Jie DONG,Luo-feng XIE,Yang-yang ZHU,Guo-fu YIN. Surface defect detection algorithm of magnetic tiles based on multi⁃branch convolutional neural network [J]. Journal of Jilin University(Engineering and Technology Edition), 2023, 53(5): 1449-1457.
[3] Ze-qiang ZHANG,Wei LIANG,Meng-ke XIE,Hong-bin ZHENG. Elite differential evolution algorithm for mixed⁃model two⁃side disassembly line balancing problem [J]. Journal of Jilin University(Engineering and Technology Edition), 2023, 53(5): 1297-1304.
[4] Peng YU,Yan PIAO. New method for extracting person re-identification attributes based on multi-scale features [J]. Journal of Jilin University(Engineering and Technology Edition), 2023, 53(4): 1155-1162.
[5] Yu JIANG,Jia-zheng PAN,He-huai CHEN,Ling-zhi FU,Hong QI. Segmentation-based detector for traditional Chinese newspaper [J]. Journal of Jilin University(Engineering and Technology Edition), 2023, 53(4): 1146-1154.
[6] Zhen-yu WU,Xiao-fei LIU,Yi-pu WANG. Trajectory planning of unmanned system based on DKRRT*⁃APF algorithm [J]. Journal of Jilin University(Engineering and Technology Edition), 2023, 53(3): 781-791.
[7] Bo TAO,Fu-wu YAN,Zhi-shuai YIN,Dong-mei WU. 3D object detection based on high⁃precision map enhancement [J]. Journal of Jilin University(Engineering and Technology Edition), 2023, 53(3): 802-809.
[8] Shan XUE,Ya-liang ZHANG,Qiong-ying LYU,Guo-hua CAO. Anti⁃unmanned aerial vehicle system object detection algorithm under complex background [J]. Journal of Jilin University(Engineering and Technology Edition), 2023, 53(3): 891-901.
[9] Hong-yang PAN,Zhao LIU,Bo YANG,Geng SUN,Yan-heng LIU. Overview of swarm intelligence methods for unmanned aerial vehicle systems based on new⁃generation information technology [J]. Journal of Jilin University(Engineering and Technology Edition), 2023, 53(3): 629-642.
[10] Ying HE,Jun-song FAN,Wei WANG,Geng SUN,Yan-heng LIU. Joint optimization of secure communication and trajectory planning in unmanned aerial vehicle air⁃to⁃ground [J]. Journal of Jilin University(Engineering and Technology Edition), 2023, 53(3): 913-922.
[11] Peng GUO,Wen-chao ZHAO,Kun LEI. Dual⁃resource constrained flexible job shop optimal scheduling based on an improved Jaya algorithm [J]. Journal of Jilin University(Engineering and Technology Edition), 2023, 53(2): 480-487.
[12] Jin-Zhen Liu,Guo-Hui Gao,Hui Xiong. Multi⁃scale attention network for brain tissue segmentation [J]. Journal of Jilin University(Engineering and Technology Edition), 2023, 53(2): 576-583.
[13] Xian-yu QI,Wei WANG,Lin WANG,Yu-fei ZHAO,Yan-peng DONG. Semantic topological map building with object semantic grid map [J]. Journal of Jilin University(Engineering and Technology Edition), 2023, 53(2): 569-575.
[14] Xiao-hu SHI,Jia-qi WU,Chun-guo WU,Shi CHENG,Xiao-hui WENG,Zhi-yong CHANG. Residual network based curve enhanced lane detection method [J]. Journal of Jilin University(Engineering and Technology Edition), 2023, 53(2): 584-592.
[15] Fu-heng QU,Tian-yu DING,Yang LU,Yong YANG,Ya-ting HU. Fast image codeword search algorithm based on neighborhood similarity [J]. Journal of Jilin University(Engineering and Technology Edition), 2022, 52(8): 1865-1871.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
Copyright © Journal of Jilin University(Engineering and Technology Edition), All Rights Reserved.
Tel: +86-431-85095297　Fax: +86-431-85094074　E-mail: xbgxb@jlu.edu.cn
Powered by Beijing Magtech Co. Ltd