基于时空注意力的多视角人脸表情识别算法

doi:10.13229/j.cnki.jdxbgxb.20240582

Abstract

Abstract:

Firstly， skin color segmentation technology was used to locate facial regions in student images， and the located facial regions were input into the spatiotemporal attention module to obtain key information from multiple perspectives of the face. Secondly， the parameters in the convolutional neural network were optimized using an adaptive gradient descent algorithm with weighted decay， and key facial information was input into the optimized network to determine the types of facial expressions of students and complete multi view facial expression recognition. The experimental results show that the proposed algorithm can accurately extract key information of the face， and the accuracy of facial expression recognition is 100%. Therefore， the proposed algorithm can effectively recognize faces and improve the accuracy of facial expression recognition.

Key words: spatiotemporal attention, facial expression recognition, skin color segmentation, facial localization, convolutional neural networks

CLC Number:

TP391.41

Rui-shan DU,Zi-shan WANG. Multi perspective facial expression recognition algorithm based on spatiotemporal attention[J].Journal of Jilin University(Engineering and Technology Edition), 2025, 55(6): 2097-2102.

Figures/Tables 5

Fig.1

Table 1

Fig.2

Fig.3

Table 2

References 15

[1]	王军杰, 王泉, 蒋平, 等. 一种孤立中心损失方法及其在人脸表情识别中的应用[J]. 西安交通大学学报, 2022, 56(4): 119-126.
	Wang Jun-jie, Wang Quan, Jiang Ping, et al. An isolated central loss method applied in facial expression recognition[J]. Journal of Xi'an Jiaotong University, 2022, 56(4): 119-126.
[2]	周丽芳, 刘俊林, 李伟生, 等. 深度二值卷积网络的人脸表情识别方法[J]. 计算机辅助设计与图形学学报, 2022, 34(3): 425-436.
	Zhou Li-fang, Liu Jun-lin, Li Wei-sheng, et al. Facial expression recognition based on deep binary convolutional network[J]. Journal of Computer-Aided Design & Computer Graphics, 2022, 34(3): 425-436.
[3]	李召峰, 朱明. 基于视频放大和双分支网络的微表情识别[J]. 液晶与显示, 2022, 37(3): 386-394.
	Li Zhao-feng, Zhu Ming. Micro-expression recognition based on video magnification and dual-branch network[J]. Chinese Journal of Liquid Crystals and Displays, 2022, 37(3): 386-394.
[4]	虞苏鑫, 贺俊吉. 基于子区域加权的不同年龄段人脸表情识别[J]. 计算机工程与科学, 2022, 44(8): 1426-1432.
	Yu Su-xin, He Jun-ji. Facial expression recognition of different age groups based on face sub-region weighting[J]. Computer Engineering & Science, 2022, 44(8): 1426-1432.
[5]	唐宏, 向俊玲, 陈海涛, 等. 多区域融合轻量级人脸表情识别网络[J]. 激光与光电子学进展, 2023, 60(6): 71-79.
	Tang Hong, Xiang Jun-ling, Chen Hai-tao, et al. Multi region fusion lightweight facial expression recognition network[J]. Progress in Laser and Optoelectronics, 2023, 60(6): 71-79.
[6]	黄兴禄, 芶小珊, 陈希. 基于混合特征与信息熵的人脸微表情识别算法[J]. 计算机仿真,2023, 40(6): 197-201.
	Huang Xing-lu, Gou Xiao-shan, Chen Xi. Face micro-expression recognition algorithm based on hybrid features and information entropy[J]. Computer Simulation, 2023, 40(6): 197-201.
[7]	戴嫣然, 戴国庆, 袁玉波. 基于肤色学习的多人脸前景抽取方法[J]. 计算机应用, 2021, 41(6): 1659-1666.
	Dai Yan-ran, Dai Guo-qing, Yuan Yu-bo. Multi-face foreground extraction method based on skin color learning[J]. Journal of Computer Applications, 2021, 41(6): 1659-1666.
[8]	王超, 刘文超, 翟海祥, 等. 基于色彩空间和暗原色先验图像融合去雾算法[J]. 电光与控制, 2022, 29(10): 44-50.
	Wang Chao, Liu Wen-chao, Zhai Hai-xiang, et al. An image fusion defogging algorithm based on color space and dark primary color priori[J]. Electronics Optics & Control, 2022, 29(10): 44-50.
[9]	朱帅康, 董龙雷, 官威, 等. 基于高斯混合模型的非高斯振动疲劳频域求解方法[J]. 振动与冲击, 2022, 41(16): 93-99.
	Zhu Shuai-kang, Dong Long-lei, Guan Wei, et al. A frequency method for fatigue life estimation under non-Gaussian random loading based on a Gaussian mixture model[J]. Journal of Vibration and Shock, 2022, 41(16): 93-99.
[10]	花胜强, 陈意, 郑慧娟, 等. 和声搜索改进的形态学分析在库区漂浮物体量预估中应用的研究[J]. 水力发电, 2022, 48(9): 108-113.
	Hua Sheng-qiang, Chen Yi, Zheng Hui-juan, et al. Research on the estimation of floating objects in the reservoir based on harmony search improved morphological analysis[J]. Water Power, 2022, 48(9): 108-113.
[11]	彭向东, 潘从成, 柯泽浚, 等. 基于并行架构和时空注意力机制的心电分类方法[J]. 浙江大学学报: 工学版, 2022, 56(10): 1912-1923.
	Peng Xiang-dong, Pan Cong-cheng, Ke Ze-jun, et al. Classification method for electrocardiograph signals based on parallel architecture model and spatiol-temporal attention mechanism[J]. Journal of Zhejiang University (Engineering Science), 2022, 56(10): 1912-1923.
[12]	张云峰, 张超, 吕钊. 基于关键点的残差全连接网络动态手势识别方法[J]. 安徽大学学报: 自然科学版, 2022, 46(2): 30-38.
	Zhang Yun-feng, Zhang Chao, Lv Zhao. Research on continuous gesture recognition based on residual fully connected network in vehicle scenes[J]. Journal of Anhui University (Natural Science Edition), 2022, 46(2): 30-38.
[13]	张蕾, 窦宏恩, 王天智, 等. 基于集成时域卷积神经网络模型的水驱油田单井产量预测方法[J]. 石油勘探与开发, 2022, 49(5): 996-1004.
	Zhang Lei, Dou Hong-en, Wang Tian-zhi, et al. A production prediction method of single well in water flooding oilfield based on integrated temporal convolutional network model[J]. Petroleum Exploration and Development, 2022, 49(5): 996-1004.
[14]	葛泉波, 张建朝, 杨秦敏, 等. 带有微分项改进的自适应梯度下降优化算法[J]. 控制理论与应用, 2022, 39(4): 623-632.
	Ge Quan-bo, Zhang Jian-chao, Yang Qin-min, et al. Adaptive gradient descent optimization algorithm with improved differential term[J]. Control Theory & Applications, 2022, 39(4): 623-632.
[15]	高涛, 杨朝晨, 陈婷, 等. 深度多尺度融合注意力残差人脸表情识别网络[J]. 智能系统学报, 2022, 17(2): 393-401.
	Gao Tao, Yang Chao-chen, Chen Ting, et al. Deep multiscale fusion attention residual network for facial expression recognition[J]. Journal of Intelligent Systems, 2022, 17(2): 393-401.

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed

Comments

Recommended 10

[1]	Nie Jian-jun，Du Fa-rong，Gao Feng . Finite time thermodynamics of real combined power cycle operating between internal combustion engine and Stirling engine with heat leak[J]. 吉林大学学报(工学版), 2007, 37(03): 518 -0523 .
[2]	Song Yu-quan，Li Da，Guan Zhi-ping，Yang Shen-shen . Curvature measuring apparatus of arbitrary shape curved surface[J]. 吉林大学学报(工学版), 2006, 36(05): 686 -0690 .
[3]	Ouyang Ji-hong，Ouyang Dan-tong，Liu Da-you . Region movement model based on fuzzy sets and RCC theory[J]. 吉林大学学报(工学版), 2007, 37(03): 591 -0594 .
[4]	Xu Tao,Zhang Yan-ning . Zero-watermarking technique of three-dimensional meshes[J]. 吉林大学学报(工学版), 2007, 37(04): 901 -904 .
[5]	Wu Yi-hua, Yang Jun-feng, He Zheng-miao，Wang Yan-fang. Clock jitter measurement technique based on signal-to-noise ratio[J]. 吉林大学学报(工学版), 2006, 36(04): 604 -607 .
[6]	Li Jing,Zuo Bin,Hu Yun-an . Time delay Elman recurrent neural network and its application in PMSM chaos control[J]. 吉林大学学报(工学版), 2008, 38(02): 460 -0465 .
[7]	Yang Yin-sheng;Sun Zhao-hua;Ma Ping;Tao Yue;Si Jin. T-S fuzzy system modeling based on hybrid of SOM and K-means [J]. 吉林大学学报(工学版), 2008, 38(03): 658 -0661 .
[8]	Zhang Ri-ming，Sun Da-wen . Fault mode and effect analysis of computer numerical control equipment [J]. 吉林大学学报(工学版), 2008, 38(增刊): 123 -0125 .
[9]	Guo Kong-hui;Lü Ji-ming;Ding Hai-tao;Guo Wen-xin . Development and implementation of MATLAB based simulation model library for vehicle components[J]. 吉林大学学报(工学版), 2006, 36(06): 866 -0870 .
[10]	Tao Hong, Zhao Mou-ming, Cui Chun . Optimization of activation and extraction of protease in hepatopancreas from Nemipterus virgatus (Houttuyn)[J]. 吉林大学学报(工学版), 2007, 37(04): 971 -975 .

参数名称	参数值
卷积核大小	3×3
卷积层数	2
池化层核大小	2×2
学习率	0.001
权重衰减	0.000 1
训练轮次	1 000

学生序号	实际类型	本文算法	文献［3］算法	文献［4］算法
1	1	1	2	1
2	3	3	2	3
3	3	3	3	2
4	2	2	2	1
5	2	2	2	2
6	3	3	3	2
7	1	1	1	1
8	1	1	1	1
9	1	1	3	1
10	3	3	3	1

Multi perspective facial expression recognition algorithm based on spatiotemporal attention

RICH HTML

PDF (PC)

Abstract

Cite this article

share this article

Figures/Tables 5

References 15

Related Articles 9

Metrics

Comments

Recommended 10

[1]	Hao WANG,Bin ZHAO,Guo-hua LIU. Temporal and motion enhancement for video action recognition [J]. Journal of Jilin University(Engineering and Technology Edition), 2025, 55(1): 339-346.
[2]	Xin-gang GUO,Chao CHENG,Zi-qi SHEN. Face expression recognition based on attention mechanism of convolution network [J]. Journal of Jilin University(Engineering and Technology Edition), 2024, 54(8): 2319-2328.
[3]	Ming-hua GAO,Can YANG. Traffic target detection method based on improved convolution neural network [J]. Journal of Jilin University(Engineering and Technology Edition), 2022, 52(6): 1353-1361.
[4]	Huai-jiang YANG,Er-shuai WANG,Yong-xin SUI,Feng YAN,Yue ZHOU. Simplified residual structure and fast deep residual networks [J]. Journal of Jilin University(Engineering and Technology Edition), 2022, 52(6): 1413-1421.
[5]	Xiang-jun LI,Jie-ying TU,Zhi-bin ZHAO. Validity classification of melting curve based on multi⁃scale fusion convolutional neural network [J]. Journal of Jilin University(Engineering and Technology Edition), 2022, 52(3): 633-639.
[6]	Hui ZHONG,Heng KANG,Ying-da LYU,Zhen-jian LI,Hong LI,Ruo-chuan OUYANG. Image manipulation localization algorithm based on channel attention convolutional neural networks [J]. Journal of Jilin University(Engineering and Technology Edition), 2021, 51(5): 1838-1844.
[7]	Hou⁃jie LI,Fa⁃sheng WANG,Jian⁃jun HE,Yu ZHOU,Wei LI,Yu⁃xuan DOU. Pseudo sample regularization Faster R⁃CNN for traffic sign detection [J]. Journal of Jilin University(Engineering and Technology Edition), 2021, 51(4): 1251-1260.
[8]	Yang LU,Shi-gang WANG,Wen-ting ZHAO,Yan ZHAO. Facial expression recognition based on separability assessment of discrete Shearlet transform [J]. Journal of Jilin University(Engineering and Technology Edition), 2019, 49(5): 1715-1725.
[9]	GUO Song, GU Guo-Chang, CA Ze-Su, LIU Hai-Bo, SHEN Jing. Face detection based on skin color segmentation and improved AdaBoostSVM algorithm [J]. 吉林大学学报(工学版), 2011, 41(02): 473-0478.