复杂环境下多模态手势关键点特征提取算法

doi:10.13229/j.cnki.jdxbgxb.20230442

摘要/Abstract

摘要：

现阶段的手势关键点特征提取在复杂背景环境下存在特征提取精度低等问题，为了解决传统方法中存在的问题，提出复杂环境下多模态手势关键点特征提取算法。首先，通过改进细菌觅食（BFO）优化算法对手势图像进行增强处理；其次，通过条件生成对抗网络对手势图像进行背景去除处理；最后，通过GIFT方法检测手势图像的关键点，并通过多尺度双树复小波变换方法和Gabor滤波方法对手势图像进行多模态手势关键点特征提取。实验结果表明：所提算法的手势关键点特征提取精度更高、效果更好。

关键词: 改进细菌觅食优化算法, 条件生成对抗网络, Gabor滤波器, 双树复小波变换, 关键点特征提取

Abstract:

At present， there are problems with low accuracy in feature extraction of gesture key points in complex background environments. In order to solve the problems existing in traditional methods， a multimodal gesture key point feature extraction algorithm research is proposed in complex environments. Firstly， the gesture image is enhanced by improving the bacterial foraging （BFO） optimization algorithm； Secondly， background removal is performed on gesture images through conditional generation of adversarial networks； Finally， the GIFT method is used to detect the key points of the gesture image， and the multimodal gesture key poiti-scale dual tree complex wavelet transform method and Gabor filtering method. The experimental results show that the proposed algorithm has higher accuracy and better performance in extracting gesture key point features.

Key words: improving bacterial foraging optimization algorithms, conditional generation adversarial network, gabor filter, double tree complex wavelet transform, key point feature extraction

中图分类号:

TP391

赖丹晖,罗伟峰,袁旭东,邱子良. 复杂环境下多模态手势关键点特征提取算法[J]. 吉林大学学报(工学版), 2024, 54(8): 2288-2294.

Dan-hui LAI,Wei-feng LUO,Xu-dong YUAN,Zi-liang QIU. Key point feature extraction algorithms for multimodal gesture in complex environments[J]. Journal of Jilin University(Engineering and Technology Edition), 2024, 54(8): 2288-2294.

图/表 6

图1

图2

图3

图4

图5

表1

参考文献 15

1	卫文韬, 李亚军. 基于双流卷积神经网络的肌电信号手势识别方法[J]. 计算机集成制造系统, 2022, 28(1): 124-131.
	Wei Wen-tao, Li Ya-jun. Surface electromyography based gesture recognition based on dual-stream CNN[J]. Computer Integrated Manufacturing Systems, 2022,28(1): 124-131.
2	王银, 陈云龙, 孙前来. 复杂背景下的手势识别[J].中国图象图形学报,2021, 26(4): 815-827.
	Wang Yin, Chen Yun-long, Sun Qian-lai. Hand gesture recognition in complex background[J]. Journal of Image and Graphics, 2021,26 (4): 815-82.
3	袁帅, 韩曼菲, 张莉莉, 等. 基于改进YOLOv3与贝叶斯分类器的手势识别方法研究[J]. 小型微型计算机系统, 2021, 42(7): 1464-1469.
	Yuan Shuai, Han Man-fei, Zhang Li-li, et al.Research approach of hand gesture recognition based on improved YOLOv3 network and bayes classifier[J]. Journal of Chinese Computer Systems, 2021, 42 (7): 1464-1469.
4	顾明, 李轶群, 张二超, 等. 可分离长短期注意力网络的手势识别方法[J]. 计算机应用, 2022, 42(): 59-63.
	Gu Ming, Li Yi-qun, Zhang Er-chao, et al. Gesture recognition method with separable long short-term attention networks[J]. Journal of Computer Applications, 2022,42 (Sup1): 59-63.
5	王婧瑶, 王红军. 基于Mask R-CNN与SG滤波的手势识别关键点特征提取方法[J]. 电子测量与仪器学报, 2021, 35(9): 41-48.
	Wang Jing-yao, Wang Hong-jun. Gesture key point extraction method based on Mask R⁃CNN and SG filter[J]. Journal of Electronic Measurement and Instrumentation, 2021,35 (9): 41-48.
6	林乐平, 卢增通, 欧阳宁. 面向非配合场景的人脸重建及识别方法[J]. 吉林大学学报: 工学版, 2022, 52(12): 2941-2946.
	Lin Le-ping, Lu Zeng-tong, Ouyang Ning.Face reconstruction and recognition in non-cooperative scenes[J].Journal of Jilin University(Engineering and Technology Edition), 2022, 52(12): 2941-2946.
7	胡振宇, 陈琦, 朱大奇. 基于颜色平衡和多尺度融合的水下图像增强[J]. 光学精密工程, 2022, 30(17):2133-2146.
	Hu Zhen-yu, Chen Qi, Zhu Da-qi. Underwater image enhancement based on color balance and multi-scale fusion[J]. Optics and Precision Engineering, 2022,30(17): 2133-2146.
8	于敏. 基于改进细菌觅食优化算法的遥感图像增强研究[J]. 激光与红外, 2022, 52(6): 931-937.
	Yu Min.Study on remote sensing image enhancement based on improved bacterial foraging algorithm[J].Laser & Infrared, 2022, 52(6): 931-937.
9	胡宇航, 胡海洋, 李忠金. 基于条件生成对抗网络的梯级表面高光去除方法[J]. 计算机应用研究,2022, 39(9): 2867-2872, 2880.
	Hu Yu-hang, Hu Hai-yang, Li Zhong-jin. Conditional generative adversarial network-based method for stepped surface highlight removal[J]. Application Research of Computers, 2022, 39(9): 2867-2872, 2880.
10	贝悦, 王琦, 程志鹏, 等.基于条件生成对抗网络的HDR图像生成方法[J].北京航空航天大学学报,2022, 48(1): 45-52.
	Bei Yue, Wang Qi, Cheng Zhi-peng, et al. HDR image generation method based on conditional generative adversarial network[J]. Journal of Beijing University of Aeronautics and Astronautics, 2022,48(1): 45-52.
11	吕晓琪, 李浩, 谷宇. 基于深度学习算法的人脸图像活体特征变换尺度提取[J]. 吉林大学学报: 工学版, 2023, 53(11): 3201-3206.
	Xiao-qi Lyu, Li Hao, Gu Yu.Adaptive blur and deduplication algorithm for digital media image based on wavelet domain[J]. Journal of Jilin University (Engineering and Technology Edition), 2023, 53(11): 3201-3206.
12	张明华, 牛玉莹, 杜艳玲, 等. 基于残差3DCNN和三维Gabor滤波器的高光谱图像分类[J]. 图学学报, 2021, 42(5): 729-737.
	Zhang Ming-hua, Niu Yu-ying, Du Yan-ling, et al. Hyperspectral image classification based on residual 3DCNN and 3D Gabor filter[J]. Journal of Graphics, 2021,42(5): 729-737.
13	吕洁, 麦雄发, 谢妙. 基于二维Gabor小波和孪生支持向量机的图像识别算法[J]. 南京理工大学学报,2022, 46(1): 113-118.
	Jie Lyu, Xiong-fa Mai, Xie Miao.Image recognition algorithm based on two-dimensional Gabor wavelet and twin support vector machine[J]. Journal of Nanjing University of Science and Technology, 2022, 46 (1): 113-118.
14	王森妹, 刘海华, 张安铎, 等. 基于Gabor卷积神经网络的图像分类算法研究[J]. 广西大学学报: 自然科学版, 2021, 46(3): 675-682.
	Wang Sen-mei, Liu Hai-hua, Zhang An-duo, et al.Research on image classification algorithm based on Gabor convolutional neural network [J].Journal of Guangxi University (Natural Science Edition), 2021,46(3): 675-682.
15	周大可, 张超, 杨欣. 基于多尺度特征融合及双重注意力机制的自监督三维人脸重建[J]. 吉林大学学报: 工学版, 2022, 52(10): 2428-2437.
	Zhou Da-ke, Zhang Chao, Yang Xin.Self-supervised 3D face reconstruction based on multi-scale feature fusion and dual attention mechanism[J].Journal of Jilin University (Engineering and Technology Edition), 2022, 52(10): 2428-2437.

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed

精确度/%
实验次数	文献［3］算法	文献［4］算法	所提算法
1	82.6	86.9	98.9
2	85.6	90.8	97.8
3	85.4	88.9	99.1
4	94.5	89.5	98.7

[1]	陈雪云,许韬,黄小巧. 基于条件生成对抗网络的医学细胞图像生成检测方法[J]. 吉林大学学报(工学版), 2021, 51(4): 1414-1419.
[2]	王玉，申铉京，陈海鹏，谭颖. 多角度特征融合的视频人脸纹理表示及识别[J]. 吉林大学学报(工学版), 2015, 45(6): 1954-1960.
[3]	李欢利, 郭立红, 王心醉, 李小明, 董月芳, 方艳超. 基于加权Gabor滤波器的虹膜识别[J]. 吉林大学学报(工学版), 2014, 44(01): 196-202.
[4]	张久文, 米进财, 张同峰. 基于双树复小波和广义高斯密度的纹理图像检索[J]. 吉林大学学报(工学版), 2013, 43(增刊1): 60-63.