吉林大学学报(工学版) ›› 2022, Vol. 52 ›› Issue (4): 725-737.doi: 10.13229/j.cnki.jdxbgxb20210088
• 综述 •
Hao-yu TIAN(),Xin MA(),Yi-bin LI
摘要:
利用低成本的Kinect相机可以实现人体姿态捕捉,代替价格昂贵的光学动作捕捉系统进行异常步态分析。本文从病理性异常步态特征、步态数据集、Kinect相机可靠性和异常步态识别方法4个方面分别对异常步态分析的发展现状展开综述。首先,总结了常见的异常步态的病理性特点,介绍了步态分析中常用的步态特征和步态事件;然后,介绍了基于Kinect相机采集的异常步态骨架数据集和可穿戴设备、压力传感器采集的异常步态数据集;广泛调查了验证Kinect用于步态分析可靠性的相关实验研究,讨论了Kinect相机及骨架数据用于步态分析的可行性;最后,分别从异常步态特征提取和异常步态分类器两个方面介绍了这一领域的发展现状,结合实际应用指出当前研究存在的不足和发展方向。
中图分类号:
1 | Chen S, Lach J, Lo B, et al. Toward pervasive gait analysis with wearable sensors: a systematic review[J]. IEEE Journal of Biomedical and Health Informatics, 2016, 20(6): 1521-1537. |
2 | Figueiredo J, Santos C P, Moreno J C. Automatic recognition of gait patterns in human motor disorders using machine learning: a review[J]. Medical Engineering & Physics, 2018, 53: 1-12. |
3 | Perry J, Davids J R. Gait analysis: normal and pathological function[J]. Journal of Pediatric Orthopaedics, 1992, 12(6): 815. |
4 | Yang G. Body sensor networks [M]. Berlin: Springer, 2006. |
5 | Herssens N, Verbecque E, Hallemans A, et al. Do spatiotemporal parameters and gait variability differ across the lifespan of healthy adults? a systematic review[J]. Gait & Posture, 2018, 64: 181-190. |
6 | 金德闻. 康复工程与生物机械学[M]. 北京:清华大学出版社, 2011. |
7 | Geerse D, Coolen B, Kolijn D, et al. Validation of foot placement locations from ankle data of a Kinect v2 sensor[J]. Sensors, 2017, 17(10): 2301. |
8 | Mentiplay B F, Perraton L G, Bower K J, et al. Gait assessment using the Microsoft Xbox One Kinect: concurrent validity and inter-day reliability of spatiotemporal and kinematic variables[J]. Journal of Biomechanics, 2015, 48(10): 2166-2170. |
9 | Springer S, Yogev Seligmann G. Validity of the kinect for gait assessment: a focused review[J]. Sensors, 2016, 16(2): 194. |
10 | Bobillo F, Dranca L, Bernad J. A fuzzy ontology-based system for gait recognition using kinect sensor[C]∥International Conference on Scalable Uncertainty Management,New York,USA, 2017:397-404. |
11 | Ravì D, Wong C, Deligianni F, et al. Deep learning for health informatics[J]. IEEE Journal of Biomedical and Health Informatics, 2016, 21(1): 4-21. |
12 | Yu K-H, Beam A L, Kohane I S. Artificial intelligence in healthcare[J]. Nature Biomedical Engineering, 2018, 2(10): 719-731. |
13 | Horst F, Lapuschkin S, Samek W, et al. Explaining the unique nature of individual gait patterns with deep learning[J]. Scientific Reports, 2019, 9(1): 1-13. |
14 | Ostchega Y, Harris T B, Hirsch R, et al. The prevalence of functional limitations and disability in older persons in the US: data from the National Health and Nutrition Examination Survey III[J]. Journal of the American Geriatrics Society, 2000, 48(9): 1132-1135. |
15 | Sudarsky L. Gait disorders: prevalence, morbidity, and etiology[J]. Advances in Neurology, 2001, 87: 111-117. |
16 | Verghese J, LeValley A, Hall C B, et al. Epidemiology of gait disorders in community‐residing older adults[J]. Journal of the American Geriatrics Society, 2006, 54(2): 255-261. |
17 | Iosa M, Fusco A, Marchetti F, et al. The golden ratio of gait harmony: repetitive proportions of repetitive gait phases[J]. BioMed Research International, 2013(3):No:918642. |
18 | Wang T, Wang Z, Zhang D, et al. Recognizing parkinsonian gait pattern by exploiting fine-grained movement function features[J]. ACM Transactions on Intelligent Systems and Technology (TIST), 2016, 8(1): 1-22. |
19 | Wang L, Tan T, Ning H, et al. Silhouette analysis-based gait recognition for human identification[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2003, 25(12): 1505-1518. |
20 | Yu S, Tan D, Tan T. A framework for evaluating the effect of view angle, clothing and carrying condition on gait recognition[C]∥The 18th International Conference on Pattern Recognition,Hong Kong, China, 2006:441-444. |
21 | Tan D, Huang K, Yu S, et al. Uniprojective features for gait recognition[C]∥International Conference on Biometrics, New Delhi, India, 2007:673-682. |
22 | Seely R D, Samangooei S, Lee M, et al. The university of southampton multi-biometric tunnel and introducing a novel 3d gait dataset[C]∥IEEE 2nd International Conference on Biometrics: Theory, Applications and Systems, Washington DC, USA,2008:1-6. |
23 | Shutler J D, Grant M G, Nixon M S, et al. On a large sequence-based human gait database[C]∥Applications and Science in Soft Computing,New York, USA, 2004:339-346. |
24 | 李贻斌, 郭佳旻, 张勤. 人体步态识别方法与技术[J]. 吉林大学学报:工学版, 2020, 50(1): 1-18. |
Li Yi-bin, Guo Jia-min, Zhang Qin. Methods and technologies of human gait recognition[J]. Journal of Jilin University (Engineering and Technology Edition), 2020, 50(1): 1-18. | |
25 | 贲晛烨, 徐森, 王科俊. 行人步态的特征表达及识别综述[J]. 模式识别与人工智能, 2012, 25(1): 71-81. |
Xian-ye Ben, Xu Sen, Wang Ke-jun. Review on pedestrian gait feature expression and recognition[J]. Pattern Recognition and Artificial Intelligence, 2012, 25(1): 71-81. | |
26 | Luo J, Tang J, Tjahjadi T, et al. Robust arbitrary view gait recognition based on parametric 3D human body reconstruction and virtual posture synthesis[J]. Pattern Recognition, 2016, 60: 361-377. |
27 | Paiement A, Tao L, Hannuna S, et al. Online quality assessment of human movement from skeleton data[C]∥British Machine Vision Conference,London,UK, 2014:153-166. |
28 | Chaaraoui A A, Padilla-López J R, Flórez-Revuelta F. Abnormal gait detection with RGB-D devices using joint motion history features[C]∥The 11th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG), Ljubljana, Slovenia,2015:1-6. |
29 | Nguyen T-N, Meunier J. Walking gait dataset: point clouds, skeletons and silhouettes[R]. Montreal:University of Montreal, 2018. |
30 | Khokhlova M, Migniot C, Morozov A, et al. Normal and pathological gait classification LSTM model[J]. Artificial Intelligence in Medicine, 2019, 94: 54-66. |
31 | Vakanski A, Jun H-P, Paul D, et al. A data set of human body movements for physical rehabilitation exercises[J]. Data, 2018, 3(1): 2. |
32 | Kastaniotis D, Theodorakopoulos I, Theoharatos C, et al. A framework for gait-based recognition using Kinect[J]. Pattern Recognition Letters, 2015, 68: 327-335. |
33 | Kastaniotis D, Theodorakopoulos I, Economou G, et al. Gait based recognition via fusing information from Euclidean and Riemannian manifolds[J]. Pattern Recognition Letters, 2016, 84: 245-251. |
34 | Nambiar A, Bernardino A, Nascimento J C, et al. Context-aware person re-identification in the wild via fusion of gait and anthropometric features[C]∥The 12th IEEE International Conference on Automatic Face & Gesture Recognition (FG), IEEE, 2017:973-980. |
35 | Wang Y, Sun J, Li J, et al. Gait recognition based on 3D skeleton joints captured by kinect[C]∥ IEEE International Conference on Image Processing (ICIP): IEEE, 2016:3151-3155. |
36 | Chen C, Jafari R, Kehtarnavaz N. UTD-MHAD: A multimodal dataset for human action recognition utilizing a depth camera and a wearable inertial sensor[C]∥IEEE International Conference on Image Processing (ICIP), IEEE, 2015:168-172. |
37 | Wang J, Liu Z, Wu Y, et al. Mining actionlet ensemble for action recognition with depth cameras[C]∥IEEE Conference on Computer Vision and Pattern Recognition,IEEE, 2012:1290-1297. |
38 | Goldberger A L, Amaral L A, Glass L, et al. PhysioBank, PhysioToolkit, and PhysioNet: components of a new research resource for complex physiologic signals[J]. Circulation, 2000, 101(23): e215-e220. |
39 | Weiss A, Brozgol M, Dorfman M, et al. Does the evaluation of gait quality during daily life provide insight into fall risk? a novel approach using 3-day accelerometer recordings[J]. Neurorehabilitation and Neural Repair, 2013, 27(8): 742-752. |
40 | Hausdorff J M, Lertratanakul A, Cudkowicz M E, et al. Dynamic markers of altered gait rhythm in amyotrophic lateral sclerosis[J]. Journal of Applied Physiology, 2000,88(6): 2045-2053.. |
41 | Frenkel-Toledo S, Giladi N, Peretz C, et al. Effect of gait speed on gait rhythmicity in Parkinson's disease: variability of stride time and swing time respond differently[J]. Journal of Neuroengineering and Rehabilitation, 2005, 2(1): 23. |
42 | Frenkel‐Toledo S, Giladi N, Peretz C, et al. Treadmill walking as an external pacemaker to improve gait rhythm and stability in Parkinson's disease[J]. Movement Disorders: Official Journal of the Movement Disorder Society, 2005, 20(9): 1109-1114. |
43 | Hausdorff J M, Lowenthal J, Herman T, et al. Rhythmic auditory stimulation modulates gait variability in Parkinson's disease[J]. European Journal of Neuroscience, 2007, 26(8): 2369-2375. |
44 | Yogev G, Giladi N, Peretz C, et al. Dual tasking, gait rhythmicity, and Parkinson's disease: which aspects of gait are attention demanding?[J]. European Journal of Neuroscience, 2005, 22(5): 1248-1256. |
45 | Bachlin M, Plotnik M, Roggen D, et al. Wearable assistant for Parkinson's disease patients with the freezing of gait symptom[J]. IEEE Transactions on Information Technology in Biomedicine, 2009, 14(2): 436-446. |
46 | Clark R A, Mentiplay B F, Hough E, et al. Three-dimensional cameras and skeleton pose tracking for physical function assessment: a review of uses, validity, current developments and Kinect alternatives[J]. Gait & Posture, 2019,68: 193-200. |
47 | Pfister A, West A M, Bronner S, et al. Comparative abilities of Microsoft Kinect and Vicon 3D motion capture for gait analysis[J]. Journal of Medical Engineering & Technology, 2014, 38(5): 274-280. |
48 | Stone E E, Skubic M. Evaluation of an inexpensive depth camera for passive in-home fall risk assessment[C]∥The 5th International Conference on Pervasive Computing Technologies for Healthcare (PervasiveHealth) and Workshops, IEEE, 2011:71-77. |
49 | Stone E E, Skubic M. Passive in-home measurement of stride-to-stride gait variability comparing vision and Kinect sensing[C]∥Annual international Conference of the IEEE Engineering in Medicine and Biology Society, IEEE, 2011:6491-6494. |
50 | Clark R A, Pua Y-H, Fortin K, et al. Validity of the Microsoft Kinect for assessment of postural control[J]. Gait & Posture, 2012, 36(3): 372-377. |
51 | Dutta T. Evaluation of the Kinect™ sensor for 3-D kinematic measurement in the workplace[J]. Applied Ergonomics, 2012, 43(4): 645-649. |
52 | Mcginley J L, Baker R, Wolfe R, et al. The reliability of three-dimensional kinematic gait measurements: a systematic review[J]. Gait & Posture, 2009, 29(3): 360-369. |
53 | Hassan E A, Jenkyn T R, Dunning C E. Direct comparison of kinematic data collected using an electromagnetic tracking system versus a digital optical system[J]. Journal of Biomechanics, 2007, 40(4): 930-935. |
54 | Galna B, Barry G, Jackson D, et al. Accuracy of the Microsoft Kinect sensor for measuring movement in people with Parkinson's disease[J]. Gait & Posture, 2014, 39(4): 1062-1068. |
55 | Guess T M, Razu S, Jahandar A, et al. Comparison of 3D joint angles measured with the kinect 2.0 skeletal tracker versus a marker-based motion capture system[J]. Journal of Applied Biomechanics, 2017, 33(2): 176-181. |
56 | Eltoukhy M, Oh J, Kuenze C, et al. Improved kinect-based spatiotemporal and kinematic treadmill gait assessment[J]. Gait & Posture, 2017, 51: 77-83. |
57 | Tanaka R, Takimoto H, Yamasaki T, et al. Validity of time series kinematical data as measured by a markerless motion capture system on a flatland for gait assessment[J]. Journal of Biomechanics, 2018, 71: 281-285. |
58 | Tölgyessy M, Dekan M, Chovanec Ľ, et al. Evaluation of the Azure Kinect and its comparison to Kinect v1 and Kinect v2[J]. Sensors, 2021, 21(2): 413. |
59 | Albert J A, Owolabi V, Gebel A, et al. Evaluation of the pose tracking performance of the azure kinect and kinect v2 for gait analysis in comparison with a gold standard: a pilot study[J]. Sensors, 2020, 20(18): 5104. |
60 | Guo Y, Deligianni F, Gu X, et al. 3-D canonical pose estimation and abnormal gait recognition with a single RGB-D camera[J]. IEEE Robotics and Automation Letters, 2019, 4(4): 3617-3624. |
61 | Cao Z, Simon T, Wei S-E, et al. Realtime multi-person 2D pose estimation using part affinity fields[C]∥Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition,Honolulu,USA,2017:7291-7299. |
62 | Halmetschlager-Funek G, Suchi M, Kampel M, et al. An empirical evaluation of ten depth cameras: Bias, precision, lateral noise, different lighting conditions and materials, and multiple sensor setups in indoor environments[J]. IEEE Robotics & Automation Magazine, 2018, 26(1): 67-77. |
63 | Alharthi A S, Yunas S U, Ozanyan K B. Deep learning for monitoring of human gait: a review[J]. IEEE Sensors Journal, 2019, 19(21): 9575-9591. |
64 | Bei S, Zhen Z, Xing Z, et al. Movement disorder detection via adaptively fused gait analysis based on kinect sensors[J]. IEEE Sensors Journal, 2018, 18(17): 7305-7314. |
65 | Li Q, Wang Y, Sharf A, et al. Classification of gait anomalies from kinect[J]. The Visual Computer, 2018, 34(2): 229-241. |
66 | Dolatabadi E, Taati B, Mihailidis A. Automated classification of pathological gait after stroke using ubiquitous sensing technology[C]∥ The 38th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC): IEEE, 2016:6150-6153. |
67 | Hu K, Wang Z, Mei S, et al. Vision-based freezing of gait detection with anatomic directed graph representation[J]. IEEE Journal of Biomedical and Health Informatics, 2019, 24(4): 1215-1225. |
68 | Procházka A, Vyšata O, Vališ M, et al. Bayesian classification and analysis of gait disorders using image and depth sensors of Microsoft Kinect[J]. Digital Signal Processing, 2015, 47: 169-177. |
69 | Ťupa O, Procházka A, Vyšata O, et al. Motion tracking and gait feature estimation for recognising Parkinson's disease using MS Kinect[J]. Biomedical Engineering Online, 2015, 14(1): 97. |
70 | Jun K, Lee D-W, Lee K, et al. Feature extraction using an RNN autoencoder for skeleton-based abnormal gait recognition[J]. IEEE Access, 2020, 8: 19196-19207. |
71 | Gu X, Guo Y, Deligianni F, et al. Cross-subject and cross-modal transfer for generalized abnormal gait pattern recognition[J]. IEEE Transactions on Neural Networks and Learning Systems, 2020,32(2): 546-560. |
72 | Tao L, Paiement A, Damen D, et al. A comparative study of pose representation and dynamics modelling for online motion quality assessment[J]. Computer Vision and Image Understanding, 2016, 148: 136-152. |
73 | Chen F, Cui X, Zhao Z, et al. Gait acquisition and analysis system for osteoarthritis based on hybrid prediction model[J]. Computerized Medical Imaging and Graphics, 2020, 85: 101782. |
74 | Kipf T N, Welling M. Semi-supervised classification with graph convolutional networks[J/OL].[2016-10-21]. . |
75 | Si C, Chen W, Wang W, et al. An attention enhanced graph convolutional lstm network for skeleton-based action recognition[C]∥Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition,Seoul, Korea, 2019:1227-1236. |
76 | Zhang P, Lan C, Zeng W, et al. Semantics-guided neural networks for efficient skeleton-based human action recognition[C]∥Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, USA, 2020:1112-1121. |
77 | Cheng K, Zhang Y, He X, et al. Skeleton-based action recognition with shift graph convolutional network[C]∥Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition,Seattle, USA, 2020:183-192. |
78 | Liu Z, Zhang H, Chen Z, et al. Disentangling and unifying graph convolutions for skeleton-based action recognition[C]∥Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition,Seattle, USA, 2020:143-152. |
79 | Ben X, Gong C, Zhang P, et al. Coupled bilinear discriminant projection for cross-view gait recognition[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2019, 30(3): 734-747. |
80 | Ben X, Zhang P, Lai Z, et al. A general tensor representation framework for cross-view gait recognition [J]. Pattern Recognition, 2019, 90: 87-98. |
81 | Li X, Makihara Y, Xu C, et al. Gait recognition via semi-supervised disentangled representation learning to identity and covariate features[C]∥Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, USA,2020:13309-13319. |
82 | Zhang Z, Tran L, Yin X, et al. Gait recognition via disentangled representation learning[C]∥Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seoul, Korea, 2019:4710-4719. |
83 | Fan C, Peng Y, Cao C, et al. Gaitpart: Temporal part-based model for gait recognition[C]∥Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, USA,2020:14225-14233. |
84 | Seifallahi M, Soltanizadeh H, Mehraban A H, et al. Alzheimer's disease detection using skeleton data recorded with Kinect camera[J]. Cluster Computing, 2020,23(2):1469-1481. |
[1] | 刘勇,徐雷,张楚晗. 面向文本游戏的深度强化学习模型[J]. 吉林大学学报(工学版), 2022, 52(3): 666-674. |
[2] | 雷景佩,欧阳丹彤,张立明. 基于知识图谱嵌入的定义域值域约束补全方法[J]. 吉林大学学报(工学版), 2022, 52(1): 154-161. |
[3] | 李志华,张烨超,詹国华. 三维水声海底地形地貌实时拼接与可视化[J]. 吉林大学学报(工学版), 2022, 52(1): 180-186. |
[4] | 欧阳丹彤,张必歌,田乃予,张立明. 结合格局检测与局部搜索的故障数据缩减方法[J]. 吉林大学学报(工学版), 2021, 51(6): 2144-2153. |
[5] | 徐艳蕾,何润,翟钰婷,赵宾,李陈孝. 基于轻量卷积网络的田间自然环境杂草识别方法[J]. 吉林大学学报(工学版), 2021, 51(6): 2304-2312. |
[6] | 杨勇,陈强,曲福恒,刘俊杰,张磊. 基于模拟划分的SP⁃k⁃means-+算法[J]. 吉林大学学报(工学版), 2021, 51(5): 1808-1816. |
[7] | 赵亚慧,杨飞扬,张振国,崔荣一. 基于强化学习和注意力机制的朝鲜语文本结构发现[J]. 吉林大学学报(工学版), 2021, 51(4): 1387-1395. |
[8] | 董延华,刘靓葳,赵靖华,李亮,解方喜. 基于BPNN在线学习预测模型的扭矩实时跟踪控制[J]. 吉林大学学报(工学版), 2021, 51(4): 1405-1413. |
[9] | 刘富,梁艺馨,侯涛,宋阳,康冰,刘云. 模糊c-harmonic均值算法在不平衡数据上改进[J]. 吉林大学学报(工学版), 2021, 51(4): 1447-1453. |
[10] | 尚福华,曹茂俊,王才志. 基于人工智能技术的局部离群数据挖掘方法[J]. 吉林大学学报(工学版), 2021, 51(2): 692-696. |
[11] | 赵海英,周伟,侯小刚,张小利. 基于多任务学习的传统服饰图像双层标注[J]. 吉林大学学报(工学版), 2021, 51(1): 293-302. |
[12] | 欧阳丹彤,马骢,雷景佩,冯莎莎. 知识图谱嵌入中的自适应筛选[J]. 吉林大学学报(工学版), 2020, 50(2): 685-691. |
[13] | 李贻斌,郭佳旻,张勤. 人体步态识别方法与技术[J]. 吉林大学学报(工学版), 2020, 50(1): 1-18. |
[14] | 徐谦,李颖,王刚. 基于深度学习的行人和车辆检测[J]. 吉林大学学报(工学版), 2019, 49(5): 1661-1667. |
[15] | 高万夫,张平,胡亮. 基于已选特征动态变化的非线性特征选择方法[J]. 吉林大学学报(工学版), 2019, 49(4): 1293-1300. |
|