基于卷积网络注意力机制的人脸表情识别

doi:10.13229/j.cnki.jdxbgxb.20221345

Abstract

Abstract:

A convolutional network based facial expression recognition method was proposed to solve the problems of large reference number and weak recognition ability in facial expression recognition. The improved residual module was introduced to reduce the parameters and enhanced the attention to the expression area； The channel-space attention mechanism was used to assign the weights of different dimensions and positions to the expression regions extracted from the network， and the subtle feature information of the key points of expression was focused on； The refinement module was used to further extract the depth feature information. In order to obtain higher accuracy， the joint loss function was introduced to increase the out-of-class distance and reduced the in-class distance to improve the accuracy of expression recognition. The experimental results showed that the average recognition rate was 63.91% and 97.98% respectively， and the parameter was 11.34 M. Compared with VGG network and residual network， the model not only improves the recognition rate but also reduces the redundant parameters.

Key words: facial expression recognition, residual module, channel-spatial attention module, refinement module

CLC Number:

TP391

Xin-gang GUO,Chao CHENG,Zi-qi SHEN. Face expression recognition based on attention mechanism of convolution network[J].Journal of Jilin University(Engineering and Technology Edition), 2024, 54(8): 2319-2328.

Figures/Tables 15

Fig.1

Fig.2

Fig.3

Fig.4

Fig.5

Table 1

Table 2

Fig.6

Fig.7

Fig.8

Table 3

Table 4

Table 5

Table 6

Table 7

References 30

1	方明, 陈文强. 结合残差网络及目标掩码的人脸微表情识别[J]. 吉林大学学报: 工学版, 2021, 51(1): 303-313.
	Fang Ming, Chen Wen-qiang. Facial micro-expression recognition based on residual error network and object mask[J]. Journal of Jilin University(Engineering and Technology Edition), 2021, 51(1): 303-313.
2	Ekman P, Friesen W V. Contants across cultures in the face and emotion[J]. Journal of Personality and Social Psychology, 1971, 17(2): 124-129.
3	郑伟鹏. 基于深度学习的人脸表情识别方法研究[D]. 桂林: 广西师范大学, 2022.
	Zhen Wei-peng. Research on facial expression recognition based on deep learning[D]. Guilin: Guangxi Normal University,2022.
4	孙旭菲, 缪新颖, 毕甜甜, 等. SFSR-Age: 一种基于人脸强语义的年龄识别算法[J]. 吉林大学学报:理学版, 2024, 62(2): 347-0356.
	Sun Xu-fei, Miao Xin-ying, Bi Tian-tian, et al. SFSR-Age: an age recognition algorithm based on strong facial semantics[J]. Journal of Jilin University (Science Edition), 2024, 62(2): 347-0356.
5	葛延良, 孙笑笑, 张乔, 等. 基于循环生成对抗网络的人脸素描合成[J]. 吉林大学学报:理学版, 2022, 60(4): 897-905.
	Ge Yan-liang, Sun Xiao-xiao, Zhang Qiao, et al. Face sketch synthesis based on cycle-generative adversarial networks[J]. Journal of Jilin University (Science Edition), 2022, 60(4): 897-905.
6	Ben X Y, Ren Y, Zhang J P, et al. Video-based facial micro-expression analysis: a survey of datasets, features and algorithms[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022, 44(9): 5826-5846.
7	贲晛烨, 杨明强, 张鹏, 等. 微表情自动识别综述[J]. 计算机辅助设计与图形学学报, 2014, 26(9): 1385-1395.
	Xian-yu Ben, Yang Ming-qiang, Zhang Peng, et al. Survey on automatic micro expression recognition methods[J]. Journal of Computer-Aided Design and Computer Graphics, 2014, 26(9): 1385-1395.
8	张超,卢韶芳,周付根. 眼部特征自动检测定位方法[J].吉林大学学报: 工学版,2015,45(5): 1717-1723.
	Zhang Chao, Lu Shao-fang, Zhou Fu-gen. Automatic detection localization method of eye feature[J]. Journal of Jilin University(Engineering and Technology Edition), 2015, 45(5): 1717-1723.
9	谢银成,黎曦,李天,等. 基于改进ResNet和损失函数的表情识别[J]. 自动化与仪表, 2022, 37(4): 64-69.
	Xie Yin-cheng, Li Xi, Li Tian, et al. Expression recognition based on improved ResNeT and loss function[J]. Automation and Instrumentation, 2022, 37(4): 64-69.
10	何超,侯明. 基于改进卷积神经网络的人脸表情识别方法[J]. 信息技术,2022,2022(5): 107-111.
	He Chao, Hou Ming. Facial expression recognition base on improved convolutional neural network[J]. Information Technology, 2022,2022(5) : 107-111.
11	崔子越, 皮家甜, 陈勇, 等. 结合改进 VGGNet 和Focal Loss的人脸表情识别[J]. 计算机工程与应用,2021,57(19): 171-178.
	Cui Zi-yue, Pi Jia-tian, Chen Yong, et al. Facial expression recognition combined with improved VGGNet and Focal Loss[J]. Computer Engineering and Applications, 2021, 57(19): 171-178.
12	亢洁,李佳伟,杨思力. 基于域适应卷积神经网络的人脸表情识别[J]. 计算机工程,2019,45(12): 201-206.
	Kang Jie, Li Jia-wei, Yang Si-li. Facial expression recognition based on convolution neural network with domain adaption[J]. Computer Engineering, 2019, 45(12): 201-206.
13	张波,兰艳亭,李大威,等. 基于卷积网络通道注意力的人脸表情识别[J]. 无线电工程,2022,52(1):148-153.
	Zhang Bo, Lan Yan-ting, Li Da-wei, et al. Face expression recognition based on convolution network channel attention[J]. Radio Engineering, 2022, 52(1): 148-153.
14	Jiang D H, Hu Y Z, Dai L, et al. Facial expression recognition based on attention mechanism[J]. Scientific Programming, 2021, 2021(1):1-5.
15	Sanghyun W, Jongchan P, Joonyoung L, et al. CBAM: convolutional block attention module[J]. European Conference on Computer Vision, 2018: 3-19.
16	张强, 彭骨, 薛陈斌. 基于改进图注意力网络的油井产量预测模型[J]. 吉林大学学报: 理学版, 2024, 62(4): 933-942.
	Zhang Qiang, Peng Gu, Xue Chen-bin. Oil well production prediction model based on improved graph attention network[J]. Journal of Jilin University (Science Edition), 2024, 62(4): 933-942.
17	余久方, 李中科, 陈涛. 基于分离混合注意力机制的人脸表情识别[J]. 电讯技术, 2022, 62(9): 1207-1214.
	Yu Jiu-fang, Li Zhong-ke, Chen Tao. Facial expression recognition based on separate hybird attention mechanism[J]. Telecommunication Engineering, 2022, 62(9): 1207-1214 .
18	梁华刚,王亚茹,张志伟. 基于Res-Bi-LSTM的人脸表情识别[J]. 计算机工程与应用, 2020, 56(13): 204-209.
	Liang Hua-gang, Wang Ya-ru, Zhang Zhi-wei. Facial expression recognition based on Res-Bi-LSTM[J]. Computer Engineering and Applications, 2020, 56(13): 204-209.
19	Sanghyun W, Jongchan P, Joonyoung L, et al. CBAM: Convolutional block attention module[C]// European Conference on Computer Vision, Munich, Germany, 2018: 3-19.
20	He K, Zhang X, Ren S, et al. Deep residual learning for image recognition[C]//Proceedings of the IEEE Computer Vision and Pattern Recognition, Nevada, USA, 2016: 770-778.
21	Lucey P, Cohn J F, Kanade T, et al. The extended cohn-kanade dataset (CK+): a complete dataset for action unit and emotion-specified expression[C]// Proceedings of 2010 IEEE Computer Society Conference on Computer Vision and Pattern Regression-Workshops, San Francisco, USA, 2010: 94-101.
22	Hu J, Shen L, Sun G. Squeeze-and-excitation networks[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, USA, 2018: 7132-7141.
23	刘全明, 辛阳阳. 端到端的低质人脸图像表情识别[J]. 小型微型计算机系统, 2020, 41(3): 668-672.
	Liu Quan-ming, Xin Yang-yang. End-to-end low-quality face image expression recognition[J]. Small Microcomputer System, 2020, 41(3): 668-672.
24	Chollet F. Xception: deep learning with depth wise separable convolutions[C]//IEEE Conference on Computer Vision and Pattern Recognition, Hawaii, USA, 2017: 1800-1807.
25	程学军, 邢萧飞. 利用改进型VGG标签学习的表情识别方法[J]. 计算机工程与设计, 2022, 43(4): 1134-1144.
	Chen Xue-jun, Xing Xiao-fei. An expression recognition method based on improved VGG tag learning[J]. Computer Engineering and Applications, 2022, 43(4):1134-1144.
26	赵家琦, 周颖玥, 王欣宇, 等. 采用支路辅助学习的人脸表情识别[J]. 计算机工程与应用, 2022, 58(23): 151-160.
	Zhao Jia-qi, Zhou Ying-yue, Wang Xin-yu, et al. Facial expression recognition using branch-assisted learning[J]. Computer Engineering and Applications. 2022, 58(23): 151-160.
27	马金峰. 基于密集连接卷积结构的人脸表情识别研究[J]. 电脑与电信, 2021, 2021(4): 1-5.
	Ma Jin-feng. Research on facial expression recognition based on dense convolution structure[J]. Computers and Telecommunications, 2021, 2021(4): 1-5.
28	高涛, 邵倩, 张亚南, 等. 基于深度残差网络的人脸表情识别研究[J]. 电子设计工程, 2020, 28 (23): 101-104.
	Gao Tao, Shao Qian, Zhang Ya-nan,et al. Research on facial expression recognition based on depth residual network[J]. Electronic Design Engineering, 2020, 28(23): 101-104.
29	刘尚旺, 刘承伟, 张爱丽. 基于深度可分离卷积神经网络的实时人脸表情和性别分类[J]. 计算机应用, 2020, 40(4): 990-995.
	Liu Shang-wang, Liu Cheng-wei, Zhang Ai-li. Real-time facial expression and gender recognition based on depthwise separable convolutional neural network[J]. 2020, 40(4): 990-995.
30	石翠萍, 谭聪, 左江, 等. 基于改进AlexNet卷积神经网络的人脸表情识别[J]. 电讯技术, 2020, 60 (9): 1005-1012.
	Shi Cui-ping, Tan Cong, Zuo Jiang, et al. Facial expression recognition based on improved AlexNet convolutional neural network[J]. Telecommunications Technology, 2020, 60(9): 1005-1012.

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed

Comments

Recommended 10

[1]	LI Shoutao, LI Yuanchun. Autonomous Mobile Robot Control Algorithm Based on Hierarchical Fuzzy Behaviors in Unknown Environments[J]. 吉林大学学报(工学版), 2005, 35(04): 391 -397 .
[2]	Liu Qing-min，Wang Long-shan，Chen Xiang-wei，Li Guo-fa. Ball nut detection by machine vision[J]. 吉林大学学报(工学版), 2006, 36(04): 534 -538 .
[3]	Li Hong-ying; Shi Wei-guang;Gan Shu-cai. Electromagnetic properties and microwave absorbing property of Z type hexaferrite Ba_3-xLa_xCo₂Fe₂₄O₄₁[J]. 吉林大学学报(工学版), 2006, 36(06): 856 -0860 .
[4]	Zhang Quan-fa，Li Ming-zhe，Sun Gang，Ge Xin . Comparison between flexible and rigid blank-holding in multi-point forming[J]. 吉林大学学报(工学版), 2007, 37(01): 25 -30 .
[5]	Yang Shu-kai, Song Chuan-xue, An Xiao-juan, Cai Zhang-lin . Analyzing effects of suspension bushing elasticity on vehicle yaw response character with virtual prototype method[J]. 吉林大学学报(工学版), 2007, 37(05): 994 -0999 .
[6]	. [J]. 吉林大学学报(工学版), 2007, 37(06): 1284 -1287 .
[7]	Che Xiang-jiu，Liu Da-you，Wang Zheng-xuan . Construction of joining surface with G¹ continuity for two NURBS surfaces[J]. 吉林大学学报(工学版), 2007, 37(04): 838 -841 .
[8]	Liu Han-bing, Jiao Yu-ling, Liang Chun-yu,Qin Wei-jun . Effect of shape function on computing precision in meshless methods[J]. 吉林大学学报(工学版), 2007, 37(03): 715 -0720 .
[9]	. [J]. 吉林大学学报(工学版), 2007, 37(04): 0 .
[10]	Li Yue-ying，Liu Yong-bing，Chen Hua . Surface hardening and tribological properties of a cam materials[J]. 吉林大学学报(工学版), 2007, 37(05): 1064 -1068 .

预处理	IHOR	CSAM	RM	JLF	FRR2013准确率/%	CK+准确率/%
48×48	×	×	×	×	60.83	95.31
48×48	√	×	×	×	62.04	96.48
48×48	×	√	×	×	62.12	96.72
48×48	√	√	×	×	62.38	97.32
48×48	×	×	√	×	61.54	95.81
48×48	√	√	√	×	62.94	97.54
48×48	×	×	×	√	62.31	96.27
64×64	√	√	√	√	62.44	96.64
48×48	√	√	√	√	63.91	97.98

CA	SA	CL	CEL	FER2013 准确率/%	CK+ 准确率/%
√	×	√	√	62.57	96.44
×	√	√	√	62.87	96.81
√	√	√	×	63.21	97.50
√	√	×	√	62.95	97.07
√	√	√	√	63.91	97.98

识别方法	准确率/%
文献［23］	62.65
文献［22］	57.13
文献［23］	63.56
VGG16	60.40
CNN	62.08
本文	63.91

识别方法	参数量/M
VGG16	22.12
Resnet50	55.73
文献［24］	22.80
文献［25］	15.40
文献［26］	26.36
文献［27］	28.50
本文模型	11.34

真实标签	预测标签
真实标签	生气	厌恶	恐惧	高兴	中性	悲伤	惊讶
生气	1.00	0.00	0.00	0.00	0.00	0.00	0.00
厌恶	0.00	0.93	0.00	0.00	0.00	0.00	0.07
恐惧	0.00	0.00	0.96	0.04	0.00	0.00	0.00
高兴	0.00	0.00	0.00	0.89	0.00	0.11	0.00
中性	0.00	0.00	0.00	0.07	0.93	0.00	0.00
悲伤	0.00	0.00	0.05	0.00	0.07	0.88	0.00
惊讶	0.12	0.00	0.00	0.00	0.00	0.07	0.81

Face expression recognition based on attention mechanism of convolution network

RICH HTML

PDF (PC)

Abstract

Cite this article

share this article

Figures/Tables 15

References 30

Related Articles 1

Metrics

Comments

Recommended 10

识别方法	准确率/%
文献［9］	95.56
文献［10］	97.56
文献［28］	93.85
文献［29］	96.00
文献［30］	97.46
Resnet 50	95.26
文献［24］	94.50
本文	97.98