复杂环境下多模态手势关键点特征提取算法

doi:10.13229/j.cnki.jdxbgxb.20230442

Abstract

Abstract:

At present， there are problems with low accuracy in feature extraction of gesture key points in complex background environments. In order to solve the problems existing in traditional methods， a multimodal gesture key point feature extraction algorithm research is proposed in complex environments. Firstly， the gesture image is enhanced by improving the bacterial foraging （BFO） optimization algorithm； Secondly， background removal is performed on gesture images through conditional generation of adversarial networks； Finally， the GIFT method is used to detect the key points of the gesture image， and the multimodal gesture key poiti-scale dual tree complex wavelet transform method and Gabor filtering method. The experimental results show that the proposed algorithm has higher accuracy and better performance in extracting gesture key point features.

Key words: improving bacterial foraging optimization algorithms, conditional generation adversarial network, gabor filter, double tree complex wavelet transform, key point feature extraction

CLC Number:

TP391

Dan-hui LAI,Wei-feng LUO,Xu-dong YUAN,Zi-liang QIU. Key point feature extraction algorithms for multimodal gesture in complex environments[J].Journal of Jilin University(Engineering and Technology Edition), 2024, 54(8): 2288-2294.

Figures/Tables 6

Fig.1

Fig.2

Fig.3

Fig.4

Fig.5

Table 1

References 15

1	卫文韬, 李亚军. 基于双流卷积神经网络的肌电信号手势识别方法[J]. 计算机集成制造系统, 2022, 28(1): 124-131.
	Wei Wen-tao, Li Ya-jun. Surface electromyography based gesture recognition based on dual-stream CNN[J]. Computer Integrated Manufacturing Systems, 2022,28(1): 124-131.
2	王银, 陈云龙, 孙前来. 复杂背景下的手势识别[J].中国图象图形学报,2021, 26(4): 815-827.
	Wang Yin, Chen Yun-long, Sun Qian-lai. Hand gesture recognition in complex background[J]. Journal of Image and Graphics, 2021,26 (4): 815-82.
3	袁帅, 韩曼菲, 张莉莉, 等. 基于改进YOLOv3与贝叶斯分类器的手势识别方法研究[J]. 小型微型计算机系统, 2021, 42(7): 1464-1469.
	Yuan Shuai, Han Man-fei, Zhang Li-li, et al.Research approach of hand gesture recognition based on improved YOLOv3 network and bayes classifier[J]. Journal of Chinese Computer Systems, 2021, 42 (7): 1464-1469.
4	顾明, 李轶群, 张二超, 等. 可分离长短期注意力网络的手势识别方法[J]. 计算机应用, 2022, 42(): 59-63.
	Gu Ming, Li Yi-qun, Zhang Er-chao, et al. Gesture recognition method with separable long short-term attention networks[J]. Journal of Computer Applications, 2022,42 (Sup1): 59-63.
5	王婧瑶, 王红军. 基于Mask R-CNN与SG滤波的手势识别关键点特征提取方法[J]. 电子测量与仪器学报, 2021, 35(9): 41-48.
	Wang Jing-yao, Wang Hong-jun. Gesture key point extraction method based on Mask R⁃CNN and SG filter[J]. Journal of Electronic Measurement and Instrumentation, 2021,35 (9): 41-48.
6	林乐平, 卢增通, 欧阳宁. 面向非配合场景的人脸重建及识别方法[J]. 吉林大学学报: 工学版, 2022, 52(12): 2941-2946.
	Lin Le-ping, Lu Zeng-tong, Ouyang Ning.Face reconstruction and recognition in non-cooperative scenes[J].Journal of Jilin University(Engineering and Technology Edition), 2022, 52(12): 2941-2946.
7	胡振宇, 陈琦, 朱大奇. 基于颜色平衡和多尺度融合的水下图像增强[J]. 光学精密工程, 2022, 30(17):2133-2146.
	Hu Zhen-yu, Chen Qi, Zhu Da-qi. Underwater image enhancement based on color balance and multi-scale fusion[J]. Optics and Precision Engineering, 2022,30(17): 2133-2146.
8	于敏. 基于改进细菌觅食优化算法的遥感图像增强研究[J]. 激光与红外, 2022, 52(6): 931-937.
	Yu Min.Study on remote sensing image enhancement based on improved bacterial foraging algorithm[J].Laser & Infrared, 2022, 52(6): 931-937.
9	胡宇航, 胡海洋, 李忠金. 基于条件生成对抗网络的梯级表面高光去除方法[J]. 计算机应用研究,2022, 39(9): 2867-2872, 2880.
	Hu Yu-hang, Hu Hai-yang, Li Zhong-jin. Conditional generative adversarial network-based method for stepped surface highlight removal[J]. Application Research of Computers, 2022, 39(9): 2867-2872, 2880.
10	贝悦, 王琦, 程志鹏, 等.基于条件生成对抗网络的HDR图像生成方法[J].北京航空航天大学学报,2022, 48(1): 45-52.
	Bei Yue, Wang Qi, Cheng Zhi-peng, et al. HDR image generation method based on conditional generative adversarial network[J]. Journal of Beijing University of Aeronautics and Astronautics, 2022,48(1): 45-52.
11	吕晓琪, 李浩, 谷宇. 基于深度学习算法的人脸图像活体特征变换尺度提取[J]. 吉林大学学报: 工学版, 2023, 53(11): 3201-3206.
	Xiao-qi Lyu, Li Hao, Gu Yu.Adaptive blur and deduplication algorithm for digital media image based on wavelet domain[J]. Journal of Jilin University (Engineering and Technology Edition), 2023, 53(11): 3201-3206.
12	张明华, 牛玉莹, 杜艳玲, 等. 基于残差3DCNN和三维Gabor滤波器的高光谱图像分类[J]. 图学学报, 2021, 42(5): 729-737.
	Zhang Ming-hua, Niu Yu-ying, Du Yan-ling, et al. Hyperspectral image classification based on residual 3DCNN and 3D Gabor filter[J]. Journal of Graphics, 2021,42(5): 729-737.
13	吕洁, 麦雄发, 谢妙. 基于二维Gabor小波和孪生支持向量机的图像识别算法[J]. 南京理工大学学报,2022, 46(1): 113-118.
	Jie Lyu, Xiong-fa Mai, Xie Miao.Image recognition algorithm based on two-dimensional Gabor wavelet and twin support vector machine[J]. Journal of Nanjing University of Science and Technology, 2022, 46 (1): 113-118.
14	王森妹, 刘海华, 张安铎, 等. 基于Gabor卷积神经网络的图像分类算法研究[J]. 广西大学学报: 自然科学版, 2021, 46(3): 675-682.
	Wang Sen-mei, Liu Hai-hua, Zhang An-duo, et al.Research on image classification algorithm based on Gabor convolutional neural network [J].Journal of Guangxi University (Natural Science Edition), 2021,46(3): 675-682.
15	周大可, 张超, 杨欣. 基于多尺度特征融合及双重注意力机制的自监督三维人脸重建[J]. 吉林大学学报: 工学版, 2022, 52(10): 2428-2437.
	Zhou Da-ke, Zhang Chao, Yang Xin.Self-supervised 3D face reconstruction based on multi-scale feature fusion and dual attention mechanism[J].Journal of Jilin University (Engineering and Technology Edition), 2022, 52(10): 2428-2437.

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed

Comments

Recommended 10

[1]	LI Shoutao, LI Yuanchun. Autonomous Mobile Robot Control Algorithm Based on Hierarchical Fuzzy Behaviors in Unknown Environments[J]. 吉林大学学报(工学版), 2005, 35(04): 391 -397 .
[2]	Liu Qing-min，Wang Long-shan，Chen Xiang-wei，Li Guo-fa. Ball nut detection by machine vision[J]. 吉林大学学报(工学版), 2006, 36(04): 534 -538 .
[3]	Li Hong-ying; Shi Wei-guang;Gan Shu-cai. Electromagnetic properties and microwave absorbing property of Z type hexaferrite Ba_3-xLa_xCo₂Fe₂₄O₄₁[J]. 吉林大学学报(工学版), 2006, 36(06): 856 -0860 .
[4]	Zhang Quan-fa，Li Ming-zhe，Sun Gang，Ge Xin . Comparison between flexible and rigid blank-holding in multi-point forming[J]. 吉林大学学报(工学版), 2007, 37(01): 25 -30 .
[5]	Yang Shu-kai, Song Chuan-xue, An Xiao-juan, Cai Zhang-lin . Analyzing effects of suspension bushing elasticity on vehicle yaw response character with virtual prototype method[J]. 吉林大学学报(工学版), 2007, 37(05): 994 -0999 .
[6]	. [J]. 吉林大学学报(工学版), 2007, 37(06): 1284 -1287 .
[7]	Che Xiang-jiu，Liu Da-you，Wang Zheng-xuan . Construction of joining surface with G¹ continuity for two NURBS surfaces[J]. 吉林大学学报(工学版), 2007, 37(04): 838 -841 .
[8]	Liu Han-bing, Jiao Yu-ling, Liang Chun-yu,Qin Wei-jun . Effect of shape function on computing precision in meshless methods[J]. 吉林大学学报(工学版), 2007, 37(03): 715 -0720 .
[9]	. [J]. 吉林大学学报(工学版), 2007, 37(04): 0 .
[10]	Li Yue-ying，Liu Yong-bing，Chen Hua . Surface hardening and tribological properties of a cam materials[J]. 吉林大学学报(工学版), 2007, 37(05): 1064 -1068 .

精确度/%
实验次数	文献［3］算法	文献［4］算法	所提算法
1	82.6	86.9	98.9
2	85.6	90.8	97.8
3	85.4	88.9	99.1
4	94.5	89.5	98.7

Key point feature extraction algorithms for multimodal gesture in complex environments

RICH HTML

PDF (PC)

Abstract

Cite this article

share this article

Figures/Tables 6

References 15

Related Articles 5

Metrics

Comments

Recommended 10

[1]	Xiao-qi LYU,Hao LI,Yu GU. Adaptive blur and deduplication algorithm for digital media image based on wavelet domain [J]. Journal of Jilin University(Engineering and Technology Edition), 2023, 53(11): 3201-3206.
[2]	En-ze LIU,Wen-fu WU. Monochrome fruit growth detection internet architecture based oncomprehensive indicator quality evaluation algorithm [J]. Journal of Jilin University(Engineering and Technology Edition), 2019, 49(6): 2019-2026.
[3]	LIU Yuan-ning, LIU Shuai, ZHU Xiao-dong, CHEN Yi-hao, ZHENG Shao-ge, SHEN Chun-zhuang. LOG operator and adaptive optimization Gabor filtering for iris recognition [J]. Journal of Jilin University(Engineering and Technology Edition), 2018, 48(5): 1606-1613.
[4]	WANG Yu, SHEN Xuan-jing, CHEN Hai-peng, TAN Ying. Video-based face texture representation and recognition with fusion features from multi-view [J]. 吉林大学学报(工学版), 2015, 45(6): 1954-1960.
[5]	LI Huan-li, GUO LI-hong, WANG Xin-zui, LI Xiao-ming, DONG Yue-fang, FANG Yan-chao. Iris recognition based on weighted Gabor filter [J]. 吉林大学学报(工学版), 2014, 44(01): 196-202.