面向非配合场景的人脸重建及识别方法

doi:10.13229/j.cnki.jdxbgxb20210389

Abstract

Abstract:

In non-cooperative scenes， to solve the problems of the poor reconstruction of faces and the low accuracy of face recognition due to the bad condition of posture deviations and the low image resolution， a triplet loss constrained generative adversarial network for reconstruction and recognition was proposed. The input faces were reconstructed firstly by encoder-decoder network， and then the reconstructed faces were recognized. In terms of training， a shortest distance triplet loss function was designed jointly with the adversarial mechanism， which increases the similarity intra-identification while enlarge the discrepancy inter-identification on the extracted feature representations. It is shown by the experiments that even under the condition of large angle deviations and low image resolution， the proposed algorithm performs better than currently leading face pose correction algorithms.

Key words: signal and information processing, face posture correction, shortest distance triplet loss, generative adversarial networks, encoder-decoder network

CLC Number:

TP753

Le-ping LIN,Zeng-tong LU,Ning OUYANG. Face reconstruction and recognition in non⁃cooperative scenes[J].Journal of Jilin University(Engineering and Technology Edition), 2022, 52(12): 2941-2946.

Figures/Tables 7

Fig.1

Fig.2

Table 1

Fig.3

Fig.4

Table 2

Table 3

References 25

1,1	齐妙, 闫光友, 徐慧, 等. 基于多尺度特征选择网络的人脸表情识别[J]. 吉林大学学报:理学版,2022, 60(2): 425-431.
	Qi Miao, Yan Guang-you, Xu Hui, et al. Facial expression recognition based on multi-scale feature selection network[J]. Journal of Jilin University (Science Edition), 2022, 60(2): 425-431..
2	Zhao J, Cheng Y, Xu Y, et al. Towards pose invariant face recognition in the wild[C]∥Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, 2018: 2207-2216.
3	Cheng J, Chen Y P P, Li M, et al. Tc-GAN: triangle cycle-consistent GANs for face frontalization with facial features preserved[C]∥Proceedings of the 27th ACM International Conference on Multimedia, Nice, France, 2019: 220-228.
4	葛延良, 孙笑笑, 张乔, 等. 基于循环生成对抗网络的人脸素描合成[J]. 吉林大学学报:理学版,2022,60(4): 897-905.
	Ge Yan-liang, Sun Xiao-xiao, Zhang Qiao, et al. Face Sketch Synthesis Based on Cycle-Generative Adversarial Networks[J]. Journal of Jilin University (Science Edition), 2022, 60(4): 897-905..
5	周大可,张超,杨欣. 基于多尺度特征融合及双重注意力机制的自监督三维人脸重建[J]. 吉林大学学报: 工学版, 2022, 52(10): 2428-2437.
	Zhou Da-ke, Zhang Chao, Yang Xin. Self-supervised 3D face reconstruction based on multi-scale feature fusion and dual attention mechanism[J]. Journal of Jilin University (Engineering and Technology Edition), 2022, 52(10): 2428-2437.
6	Huang R, Zhang S, Li T, et al. Beyond face rotation: global and local perception gan for photorealistic and identity preserving frontal view synthesis[C]∥Proceedings of the IEEE International Conference on Computer Vision, Honolulu, HI, USA, 2017: 2439-2448.
7	Huang H, Ran H, Sun Z, et al. Wavelet-SRNet: a wavelet-based cnn for multi-scale face super resolution[C]∥Proceedings of the IEEE International Conference on Computer Vision, Honolulu, HI, USA, 2017: 1698-1706.
8	Shi W, Caballoa J, Huszar F, et al. Real-time single image and video super-resolution using an efficient sub-pixel Convolutional Neural Network[C]∥Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA, 2016: 1874-1883.
9	Chen Y, Tai Y, Liu X, et al. Fsrnet: end-to-end learning face super-resolution with facial priors[C]∥Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, 2018: 2492-2501.
10	Goodfellow I J, Pought-Abadie J, Mirza M, et al. Generative adversarial nets[C]∥Proceedings of the 27th International Conference on Neural Information Processing Systems, Montreal, Canada, 2014: 2672-2680.
11	Bulat A, Tzimiropoulos G. Super-fan: integrated facial landmark localization and super-resolution of real-world low resolution faces in arbitrary poses with gans[C]∥Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, 2018: 109-117.
12	Li P, Wu X, Hu Y, et al. M2FPA: a multi-yaw multi-pitch high-quality dataset and benchmark for facial pose analysis[C]∥Proceedings of the IEEE International Conference on Computer Vision, Long Beach, CA, USA, 2019: 10043-10051.
13	Wang Z, Zheng L, Li Y, et al. Linkage based face clustering via graph convolution network[C]∥Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA, 2019: 1117-1125.
14	Cao K, Rong Y, Li C, et al. Pose-robust face recognition via deep residual equivariant mapping[C]∥Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, 2018: 5187-5196.
15	Zhou H, Liu J, Liu Z, et al. Rotate-and-render: unsupervised photorealistic face rotation from single-view images[C]∥Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA, 2020: 5911-5920.
16	Schroff F, Kalenichenko D, Philbin J. Facenet: a unified embedding for face recognition and clustering[C]∥Proceedings of the IEEE Conference on computer Vision and Pattern Recognition, Boston, MA, USA, 2015: 815-823.
17	Yin Y, Jiang S, Robinson J P, et al. Dual-attention GAN for large-pose face frontalization[C]∥15th IEEE International Conference on Automatic Face and Gesture Recognition, Buenos Aires, Argentina, 2020: 249-256.
18	柯鹏飞, 蔡茂国, 吴涛. 基于改进卷积神经网络与集成学习的人脸识别算法[J]. 计算机工程, 2020, 46(2): 262-267, 273.
	Ke Peng-fei, Cai Mao-guo, Wu Tao. Face recognition algorithm based on improved convolutional neural network and ensemble learning[J]. Computer Engineering, 2020, 46(2): 262-267, 273.
19	黄义妨, 魏丹丹, 武淼, 等. 面向不同传感器与复杂场景的人脸识别系统防伪方法综述[J]. 计算机工程, 2021, 47(12): 1-18.
	Huang Yi-fang, Wei Dan-dan, Wu Miao, et al. Overview of anti-spoofing methods of face recognition systems for different sensors and complex scenes[J]. Computer Engineering, 2021, 47(12): 1-18.
20	李新春,马红艳,林森.基于局部邻域四值模式的掌纹掌脉融合识别[J].重庆邮电大学学报: 自然科学版, 2020, 32(4): 630-638.
	Li Xin-chun, Ma Hong-yan, Lin Sen. Palmprint and palm vein fusion recognition based on local neighbor quaternary pattern[J]. Journal of Chongqing University of Posts and Telecommunications (Natural Science Edition), 2021, 47(5): 285-291, 300.
21	陶施帆, 李玉峰, 黄煜峰, 等. 基于深度残差网络和注意力机制的人脸检测算法[J]. 计算机工程, 2021, 47(11): 276-282.
	Tao Shi-fan, Li Yu-feng, Huang Yu-feng, et al. Face detection algorithm based on deep residual network and attention mechanism[J]. Computer Engineering, 2021, 47(11): 276-282.
22	胡静,陶洋.基于RPCA的群稀疏表示人脸识别方法[J]. 重庆邮电大学学报: 自然科学版, 2020, 32(3): 459-468.
	Hu Jing, Tao Yang. Group sparse representation face recognition method based on RPCA[J]. Journal of Chongqing University of Posts and Telecommunications (Natural Science Edition), 2020, 32(3): 459-468.
23	Wu X, He R, Sun Z, et al. A light CNN for deep face representation with noisy labels[J]. IEEE Transactions on Information Forensics and Security, 2018, 13(11): 2884-2896.
24	Cao Q, Shen L, Xie W, et al. Vggface2: a dataset for recognising faces across pose and age[C]∥13th IEEE International Conference on Automatic Face and Gesture Recognition, Xi'an, China, 2018: 67-74.
25	Arjovsky M, Chintala S, Bottou L. Wasserstein generative adversarial networks[C]∥Proceedings of International Conference on Machine Learning, Sydney, Australia, 2017: 214-223.

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed

Comments

Recommended 10

[1]	Tang Xin-xing，Zhao Ding-xuan，Huang Hai-dong，Ai Xue-zhong,Feng Shi-zhu . Modified stereo vision calibration method for construction robot [J]. 吉林大学学报(工学版), 2007, 37(02): 391 -0395 .
[2]	Zeng Ping, Liu Yan-tao, Wu Bo-da, Cheng Guang-ming，Yang Zhi-gang，Kan Jun-wu . A novel wireless electropult powered by piezoelectricity[J]. 吉林大学学报(工学版), 2006, 36(增刊2): 78 -82 .
[3]	Lu Shou-feng，Yang Zhao-sheng，Liu Xi-min . Synergy of traffic signal control and route guidance based on multiagent system[J]. 吉林大学学报(工学版), 2006, 36(增刊2): 143 -146 .
[4]	Gao Ji-dong, Zhang Yuan-jun, Li Meng-liang,Qin Kong-jian ,Chen Jie-feng . Size distributions of PM emissions from heavy-duty diesel vehicles on road[J]. 吉林大学学报(工学版), 2008, 38(01): 37 -041 .
[5]	CHENG Yong-chun，TAN Guo-jin，LIU Han-bing，FU Cong . Damage identification of bridge structure based on statistical properties of eigen-solution[J]. 吉林大学学报(工学版), 2008, 38(04): 812 -816 .
[6]	Liu Feng,Wang Xin-wei . Differential quadrature method for nonlinear buckling analysis of tubing in straight wells [J]. 吉林大学学报(工学版), 2007, 37(01): 234 -238 .
[7]	Gao Ying, Yu Xiu-min, Li Jun, Yang Shi-chun. Experiments on temperature field and conversion efficiency of three way catalyst under different engine operation conditions[J]. 吉林大学学报(工学版), 2006, 36(04): 488 -492 .
[8]	JI Lin-Da-Xue-Xue-Bao-（Gong-XueBan-）Bian-Ji-Bu. [J]. 吉林大学学报(工学版), 2010, 40(03): 1 -0004 .
[9]	WANG Hui-Hui, SUN Yong-Hai, LIU Jing-Jing, ZHANG Ting-Ting, WANG Xiao-Dan, FANG Xu-Jun. Evaluation of maturity grading of fresh corn ears based on twodimensional discrete wavelet[J]. 吉林大学学报(工学版), 2011, 41(02): 574 -0578 .
[10]	GAO Feng-Li, GUO Shu-Xu, ZHANG Zhen-Guo, YU Si-Yao, CAO Jun-Sheng, ZHANG Shuang. Denoising method applied to electrical derivative data of semiconductor laser diode[J]. 吉林大学学报(工学版), 2009, 39(05): 1288 -1291 .

卷积类型	二维卷积
Conv1.x	（3×3，32），S2
Conv2.x	（3×3，64），S2
Conv3.x	（3×3，128），S2
Conv3.x	（3×3，256），S2
FC	4096

方法	偏姿角度
方法	±90°	±75°	±60°	±45°	±30°	±15°
TP-GAN	0.5702	0.5855	0.6017	0.6273	0.6545	0.6999
FNM	0.4556	0.5478	0.5816	0.6014	0.6358	0.6852
TL-GAN	0.5950	0.6177	0.6207	0.6489	0.6550	0.7006

方法	偏姿角度
方法	±90°	±75°	±60°	±45°	±30°	±15°	Avg
Light-CNN	2.00	10.20	30.61	69.39	85.71	87.75	47.61
TP-GAN+Light-CNN	54.64	67.43	77.72	85.38	88.06	88.68	76.99
FNM+Light-CNN	61.20	77.20	85.20	89.70	92.50	94.60	83.40
TL-GAN+Light-CNN	71.43	85.71	91.83	93.87	95.91	97.95	89.79
VGG-Face	4.10	12.24	32.65	71.42	87.75	89.79	49.65
TL-GAN+VGG-Face	73.46	87.75	93.87	94.89	96.93	97.95	90.80

Face reconstruction and recognition in non⁃cooperative scenes

RICH HTML

PDF (PC)

Abstract

Cite this article

share this article

Figures/Tables 7

References 25

Related Articles 5

Metrics

Comments

Recommended 10

[1]	Xue-mei LI,Chun-yang WANG,Xue-lian LIU,Chun-hao SHI,Guo-rui LI. Point cloud registration method based on supervoxel bidirectional nearest neighbor distance ratio [J]. Journal of Jilin University(Engineering and Technology Edition), 2022, 52(8): 1918-1925.
[2]	Xue-mei LI,Chun-yang WANG,Xue-lian LIU,Da XIE. Time delay estimation of linear frequency-modulated continuous-wave lidar signals via SESTH [J]. Journal of Jilin University(Engineering and Technology Edition), 2022, 52(4): 950-958.
[3]	Hui-jing DOU,Gang DING,Jia GAO,Xiao LIANG. Wideband signal direction of arrival estimation based on compressed sensing theory [J]. Journal of Jilin University(Engineering and Technology Edition), 2021, 51(6): 2237-2245.
[4]	Xin-yu JIN,Mu-han XIE, SUN-Bin. Grain information compressed sensing based on semi-tensor product approach [J]. Journal of Jilin University(Engineering and Technology Edition), 2021, 51(1): 379-385.
[5]	Li⁃min GUO,Xin CHEN,Tao CHEN. Radar signal modulation type recognition based on AlexNet model [J]. Journal of Jilin University(Engineering and Technology Edition), 2019, 49(3): 1000-1008.