面向非配合场景的人脸重建及识别方法

doi:10.13229/j.cnki.jdxbgxb20210389

吉林大学学报(工学版) ›› 2022, Vol. 52 ›› Issue (12): 2941-2946.doi: 10.13229/j.cnki.jdxbgxb20210389

面向非配合场景的人脸重建及识别方法

林乐平^1,²(),卢增通²,欧阳宁^1,²()

^1.桂林电子科技大学认知无线电与信息处理省部共建教育部重点实验室，广西桂林 541004
^2.桂林电子科技大学信息与通信学院，广西桂林 541004

收稿日期:2021-04-30 出版日期:2022-12-01 发布日期:2022-12-08
通讯作者: 欧阳宁 E-mail:linleping@guet.edu.cn;ynou@guet.edu.cn
作者简介:林乐平（1980-），女，副教授，博士. 研究方向：机器学习，图像信号处理. E-mail：linleping@guet.edu.cn
基金资助:
国家自然科学基金项目(62001133);广西科技重大专项项目(桂科AA20302001);广西科技基地和人才专项项目(桂科AD19110060);广西无线宽带通信与信号处理重点实验室基金项目(GXKL06200114)

Face reconstruction and recognition in non⁃cooperative scenes

Le-ping LIN^1,²(),Zeng-tong LU²,Ning OUYANG^1,²()

^1.Ministry of Education Key Laboratory of Cognitive Radio and Information Processing，Guilin University of Electronic Technology，Guilin 541004，China
^2.School of Information and Communication，Guilin University of Electronic Technology，Guilin 541004，China

Received:2021-04-30 Online:2022-12-01 Published:2022-12-08
Contact: Ning OUYANG E-mail:linleping@guet.edu.cn;ynou@guet.edu.cn

摘要/Abstract

摘要：

针对非配合场景下，姿态偏差及低分辨率条件下人脸重建效果差、识别率低等问题，提出一种三元对抗重建识别网络。该方法先通过编解码网络将人脸图像重建为高分辨率正脸图像，再对重建人脸进行识别。在网络训练上，设计了一种最短距离三元组损失函数，并将其与对抗机制结合，使网络对同一个人提取的特征相似性高，而与其他人提取出的特征相似度低。实验结果表明，在大角度低分辨率条件下，本文网络性能优于目前领先的人脸姿态矫正算法。

关键词: 信号与信息处理, 人脸姿态矫正, 最短距离三元组损失, 生成对抗网络, 编解码网络

Abstract:

In non-cooperative scenes， to solve the problems of the poor reconstruction of faces and the low accuracy of face recognition due to the bad condition of posture deviations and the low image resolution， a triplet loss constrained generative adversarial network for reconstruction and recognition was proposed. The input faces were reconstructed firstly by encoder-decoder network， and then the reconstructed faces were recognized. In terms of training， a shortest distance triplet loss function was designed jointly with the adversarial mechanism， which increases the similarity intra-identification while enlarge the discrepancy inter-identification on the extracted feature representations. It is shown by the experiments that even under the condition of large angle deviations and low image resolution， the proposed algorithm performs better than currently leading face pose correction algorithms.

Key words: signal and information processing, face posture correction, shortest distance triplet loss, generative adversarial networks, encoder-decoder network

中图分类号:

TP753

林乐平,卢增通,欧阳宁. 面向非配合场景的人脸重建及识别方法[J]. 吉林大学学报(工学版), 2022, 52(12): 2941-2946.

Le-ping LIN,Zeng-tong LU,Ning OUYANG. Face reconstruction and recognition in non⁃cooperative scenes[J]. Journal of Jilin University(Engineering and Technology Edition), 2022, 52(12): 2941-2946.

图/表 7

图1

图2

表1

图3

图4

表2

表3

参考文献 25

1,1	齐妙, 闫光友, 徐慧, 等. 基于多尺度特征选择网络的人脸表情识别[J]. 吉林大学学报:理学版,2022, 60(2): 425-431.
	Qi Miao, Yan Guang-you, Xu Hui, et al. Facial expression recognition based on multi-scale feature selection network[J]. Journal of Jilin University (Science Edition), 2022, 60(2): 425-431..
2	Zhao J, Cheng Y, Xu Y, et al. Towards pose invariant face recognition in the wild[C]∥Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, 2018: 2207-2216.
3	Cheng J, Chen Y P P, Li M, et al. Tc-GAN: triangle cycle-consistent GANs for face frontalization with facial features preserved[C]∥Proceedings of the 27th ACM International Conference on Multimedia, Nice, France, 2019: 220-228.
4	葛延良, 孙笑笑, 张乔, 等. 基于循环生成对抗网络的人脸素描合成[J]. 吉林大学学报:理学版,2022,60(4): 897-905.
	Ge Yan-liang, Sun Xiao-xiao, Zhang Qiao, et al. Face Sketch Synthesis Based on Cycle-Generative Adversarial Networks[J]. Journal of Jilin University (Science Edition), 2022, 60(4): 897-905..
5	周大可,张超,杨欣. 基于多尺度特征融合及双重注意力机制的自监督三维人脸重建[J]. 吉林大学学报: 工学版, 2022, 52(10): 2428-2437.
	Zhou Da-ke, Zhang Chao, Yang Xin. Self-supervised 3D face reconstruction based on multi-scale feature fusion and dual attention mechanism[J]. Journal of Jilin University (Engineering and Technology Edition), 2022, 52(10): 2428-2437.
6	Huang R, Zhang S, Li T, et al. Beyond face rotation: global and local perception gan for photorealistic and identity preserving frontal view synthesis[C]∥Proceedings of the IEEE International Conference on Computer Vision, Honolulu, HI, USA, 2017: 2439-2448.
7	Huang H, Ran H, Sun Z, et al. Wavelet-SRNet: a wavelet-based cnn for multi-scale face super resolution[C]∥Proceedings of the IEEE International Conference on Computer Vision, Honolulu, HI, USA, 2017: 1698-1706.
8	Shi W, Caballoa J, Huszar F, et al. Real-time single image and video super-resolution using an efficient sub-pixel Convolutional Neural Network[C]∥Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA, 2016: 1874-1883.
9	Chen Y, Tai Y, Liu X, et al. Fsrnet: end-to-end learning face super-resolution with facial priors[C]∥Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, 2018: 2492-2501.
10	Goodfellow I J, Pought-Abadie J, Mirza M, et al. Generative adversarial nets[C]∥Proceedings of the 27th International Conference on Neural Information Processing Systems, Montreal, Canada, 2014: 2672-2680.
11	Bulat A, Tzimiropoulos G. Super-fan: integrated facial landmark localization and super-resolution of real-world low resolution faces in arbitrary poses with gans[C]∥Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, 2018: 109-117.
12	Li P, Wu X, Hu Y, et al. M2FPA: a multi-yaw multi-pitch high-quality dataset and benchmark for facial pose analysis[C]∥Proceedings of the IEEE International Conference on Computer Vision, Long Beach, CA, USA, 2019: 10043-10051.
13	Wang Z, Zheng L, Li Y, et al. Linkage based face clustering via graph convolution network[C]∥Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA, 2019: 1117-1125.
14	Cao K, Rong Y, Li C, et al. Pose-robust face recognition via deep residual equivariant mapping[C]∥Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, 2018: 5187-5196.
15	Zhou H, Liu J, Liu Z, et al. Rotate-and-render: unsupervised photorealistic face rotation from single-view images[C]∥Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA, 2020: 5911-5920.
16	Schroff F, Kalenichenko D, Philbin J. Facenet: a unified embedding for face recognition and clustering[C]∥Proceedings of the IEEE Conference on computer Vision and Pattern Recognition, Boston, MA, USA, 2015: 815-823.
17	Yin Y, Jiang S, Robinson J P, et al. Dual-attention GAN for large-pose face frontalization[C]∥15th IEEE International Conference on Automatic Face and Gesture Recognition, Buenos Aires, Argentina, 2020: 249-256.
18	柯鹏飞, 蔡茂国, 吴涛. 基于改进卷积神经网络与集成学习的人脸识别算法[J]. 计算机工程, 2020, 46(2): 262-267, 273.
	Ke Peng-fei, Cai Mao-guo, Wu Tao. Face recognition algorithm based on improved convolutional neural network and ensemble learning[J]. Computer Engineering, 2020, 46(2): 262-267, 273.
19	黄义妨, 魏丹丹, 武淼, 等. 面向不同传感器与复杂场景的人脸识别系统防伪方法综述[J]. 计算机工程, 2021, 47(12): 1-18.
	Huang Yi-fang, Wei Dan-dan, Wu Miao, et al. Overview of anti-spoofing methods of face recognition systems for different sensors and complex scenes[J]. Computer Engineering, 2021, 47(12): 1-18.
20	李新春,马红艳,林森.基于局部邻域四值模式的掌纹掌脉融合识别[J].重庆邮电大学学报: 自然科学版, 2020, 32(4): 630-638.
	Li Xin-chun, Ma Hong-yan, Lin Sen. Palmprint and palm vein fusion recognition based on local neighbor quaternary pattern[J]. Journal of Chongqing University of Posts and Telecommunications (Natural Science Edition), 2021, 47(5): 285-291, 300.
21	陶施帆, 李玉峰, 黄煜峰, 等. 基于深度残差网络和注意力机制的人脸检测算法[J]. 计算机工程, 2021, 47(11): 276-282.
	Tao Shi-fan, Li Yu-feng, Huang Yu-feng, et al. Face detection algorithm based on deep residual network and attention mechanism[J]. Computer Engineering, 2021, 47(11): 276-282.
22	胡静,陶洋.基于RPCA的群稀疏表示人脸识别方法[J]. 重庆邮电大学学报: 自然科学版, 2020, 32(3): 459-468.
	Hu Jing, Tao Yang. Group sparse representation face recognition method based on RPCA[J]. Journal of Chongqing University of Posts and Telecommunications (Natural Science Edition), 2020, 32(3): 459-468.
23	Wu X, He R, Sun Z, et al. A light CNN for deep face representation with noisy labels[J]. IEEE Transactions on Information Forensics and Security, 2018, 13(11): 2884-2896.
24	Cao Q, Shen L, Xie W, et al. Vggface2: a dataset for recognising faces across pose and age[C]∥13th IEEE International Conference on Automatic Face and Gesture Recognition, Xi'an, China, 2018: 67-74.
25	Arjovsky M, Chintala S, Bottou L. Wasserstein generative adversarial networks[C]∥Proceedings of International Conference on Machine Learning, Sydney, Australia, 2017: 214-223.

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed

卷积类型	二维卷积
Conv1.x	（3×3，32），S2
Conv2.x	（3×3，64），S2
Conv3.x	（3×3，128），S2
Conv3.x	（3×3，256），S2
FC	4096

方法	偏姿角度
方法	±90°	±75°	±60°	±45°	±30°	±15°
TP-GAN	0.5702	0.5855	0.6017	0.6273	0.6545	0.6999
FNM	0.4556	0.5478	0.5816	0.6014	0.6358	0.6852
TL-GAN	0.5950	0.6177	0.6207	0.6489	0.6550	0.7006

方法	偏姿角度
方法	±90°	±75°	±60°	±45°	±30°	±15°	Avg
Light-CNN	2.00	10.20	30.61	69.39	85.71	87.75	47.61
TP-GAN+Light-CNN	54.64	67.43	77.72	85.38	88.06	88.68	76.99
FNM+Light-CNN	61.20	77.20	85.20	89.70	92.50	94.60	83.40
TL-GAN+Light-CNN	71.43	85.71	91.83	93.87	95.91	97.95	89.79
VGG-Face	4.10	12.24	32.65	71.42	87.75	89.79	49.65
TL-GAN+VGG-Face	73.46	87.75	93.87	94.89	96.93	97.95	90.80

[1]	李雪梅,王春阳,刘雪莲,施春浩,李国瑞. 基于超体素双向最近邻距离比的点云配准方法[J]. 吉林大学学报(工学版), 2022, 52(8): 1918-1925.
[2]	李雪梅,王春阳,刘雪莲,谢达. 基于SESTH的线性调频连续波激光雷达信号时延估计[J]. 吉林大学学报(工学版), 2022, 52(4): 950-958.
[3]	窦慧晶,丁钢,高佳,梁霄. 基于压缩感知理论的宽带信号波达方向估计[J]. 吉林大学学报(工学版), 2021, 51(6): 2237-2245.
[4]	陈雪云,许韬,黄小巧. 基于条件生成对抗网络的医学细胞图像生成检测方法[J]. 吉林大学学报(工学版), 2021, 51(4): 1414-1419.
[5]	李锦青,周健,底晓强. 基于循环生成对抗网络的学习型光学图像加密方案[J]. 吉林大学学报(工学版), 2021, 51(3): 1060-1066.
[6]	金心宇,谢慕寒,孙斌. 基于半张量积压缩感知的粮情信息采集[J]. 吉林大学学报(工学版), 2021, 51(1): 379-385.
[7]	王小玉,胡鑫豪,韩昌林. 基于生成对抗网络的人脸铅笔画算法[J]. 吉林大学学报(工学版), 2021, 51(1): 285-292.
[8]	史再峰,李金卓,曹清洁,李慧龙,胡起星. 基于生成对抗网络的低剂量能谱层析成像去噪算法[J]. 吉林大学学报(工学版), 2020, 50(5): 1755-1764.
[9]	郭立民,陈鑫,陈涛. 基于AlexNet模型的雷达信号调制类型识别[J]. 吉林大学学报(工学版), 2019, 49(3): 1000-1008.

面向非配合场景的人脸重建及识别方法

Face reconstruction and recognition in non⁃cooperative scenes

RICH HTML

PDF (PC)

摘要/Abstract

引用本文

使用本文

图/表 7

参考文献 25

相关文章 9

Metrics

本文评价

推荐阅读 10