吉林大学学报(工学版) ›› 2025, Vol. 55 ›› Issue (1): 333-338.doi: 10.13229/j.cnki.jdxbgxb.20231349

• 计算机科学与技术 • 上一篇    

基于生成对抗网络的人脸超分辨率重建方法

张曦(),库少平()   

  1. 武汉理工大学 计算机与人工智能学院,武汉 430070
  • 收稿日期:2023-12-05 出版日期:2025-01-01 发布日期:2025-03-28
  • 通讯作者: 库少平 E-mail:zhangxi20231111@163.com;sissi810588969@126.com
  • 作者简介:张曦(1995-),女,博士研究生.研究方向:计算机动画创作,数字媒体交互设计.E-mail: zhangxi20231111@163.com
  • 基金资助:
    工业与信息化部重大专项项目(工信部装函[2017]614号)

Facial super-resolution reconstruction method based on generative adversarial networks

Xi ZHANG(),Shao-ping KU()   

  1. School of Computer Science and Artificial Intelligence,Wuhan University of Technology,Wuhan 430070,China
  • Received:2023-12-05 Online:2025-01-01 Published:2025-03-28
  • Contact: Shao-ping KU E-mail:zhangxi20231111@163.com;sissi810588969@126.com

摘要:

针对人脸重建图像峰值信噪比和结构相似度低的问题,提出了一种在生成对抗网络基础上的人脸超分辨率重建方法。创建一个两分支生成器网络,运用网络内的特征提取模块得到图像浅层特征,重建图像高频分支,利用深层残差网络训练低分辨率图像高频数据,映射人脸图像数据;停止更新固定生成器参数,使用相对判别器网络辨别人脸图像重建真伪,引入注意力机制优化生成对抗网络重建性能,通过感知损失函数评估重建可靠性,实现人脸超分辨率重建。实验结果表明:本文方法能有效提升面部细节重建能力,主客观评价指标都较为优秀,其中主观评价结果显示该方法重建的人脸图像具备更清晰的细节数据,人脸轮廓更为立体,失真率较低,拥有更为自然和真实的外在特征;客观评价结果显示该方法的峰值信噪比达到44.6 dB,结构相似度达到0.74,充分说明该方法具有较强的鲁棒性,可为优化人脸识别应用性能提供有效借鉴。

关键词: 生成对抗网络, 高频分量, 超分辨率, 判别器, 损失函数

Abstract:

In order to solve the problem of low peak signal-to-noise ratio and structural similarity of reconstructed face images, a super-resolution reconstruction method based on generative adversary-network is proposed. A two-branch generator network is created, the shallow features of the image are obtained by using the feature extraction module in the network, the high-frequency branches of the image are reconstructed, the high-frequency data of the low-resolution image is trained by the deep residual network, and the face image data is mapped. Stop updating fixed generator parameters, use relative discriminator network to identify the authenticity of face image reconstruction, introduce attention mechanism to optimize the reconstruction performance of generative adversarial network, evaluate the reconstruction reliability by perception loss function, and realize the super-resolution reconstruction of face. The experimental results show that the proposed method can effectively improve the ability of facial detail reconstruction, and the subjective and objective evaluation indexes are excellent. The subjective evaluation results show that the reconstructed facial images with the proposed method have clearer detailed data, more three-dimensional facial contours, less distortion rate, and more natural and real external features. The objective evaluation results show that the peak signal-to-noise ratio of the proposed method reaches 44.6dB and the structural similarity reaches 0.74, which fully demonstrates the robustness of the proposed method and provides an effective reference for optimizing the application performance of face recognition.

Key words: generate adversarial networks, high frequency component, super resolution, discriminator, loss function

中图分类号: 

  • TP391

图1

原始图像"

图2

不同方法人脸超分辨率重建对比"

图3

不同方法人脸重建峰值信噪比对比"

图4

不同方法人脸重建结构相似度对比"

1 周燕, 项杨, 曾凡智, 等. 超分辨率重建的微小人脸识别算法[J].小型微型计算机系统, 2020, 41(11):2340-2346.
Zhou Yan, Xiang Yang, Zeng Fan-zhi, et al.Tiny face recognition algorithm based on super-resolution reconstruction[J]. Journal of Chinese Computer Systems, 2020,41(11):2340-2346.
2 李博. 基于视觉传达的多帧图像高分辨率重建仿真[J]. 计算机仿真, 2021, 38(3): 113-116, 121.
Li Bo. High-resolution reconstruction simulation of multi-frame image based on visual communication [J]. Computer Simulation, 2021, 38(3): 113-116, 121.
3 陆要要, 袁家斌, 何珊, 等. 基于超分辨率重建的低质量视频人脸识别方法[J]. 计算机科学, 2021, 48(): 295-302.
Lu Yao-yao, Yuan Jia-bin, He Shan, et al.Low-quality video face recognition method based on super-resolution reconstruction[J]. Computer Science, 2021, 48(Sup.2): 295-302.
4 夏颖, 盖绍彦, 达飞鹏. 双目视觉下基于区域生长的三维人脸重建算法[J]. 计算机应用研究, 2021, 38(3): 932-936.
Xia Ying, Gai Shao-yan, Fei-peng Da.3D face reconstruction algorithm based on region growth in binocular vision[J].Application Research of Computers, 2021,38(3):932-936.
5 周敏, 叶钧, 杨祥. 基于形变模型和随机照片集的人脸三维重建[J]. 控制工程, 2021, 28(7): 1460-1465.
Zhou Min, Ye Jun, Yang Xiang. Face 3D reconstruction method based on morphable model and random photo datesets[J]. Control Engineering of China, 2021, 28(7): 1460-1465.
6 袁健,李佳慧.融合先验信息的残差空间注意力人脸超分辨率重建模型[J].小型微型计算机系统, 2023, 44(5): 1035-1042.
Yuan Jian, Li Jia-hui. A super-resolution reconstruction model of residual space attention based on prior information[J]. Journal of Chinese Computer Systems, 2023, 44(5): 1035-1042.
7 Xu D D, Wang X, Xu Z B,et al.A super-resolution reconstruction algorithm for two-photon fluorescence polarization microscopy[J].Optics Communications, 2021, 44(30): 116-127.
8 Sood Rewa R, Shao W, Kunder C,et al.3D registration of pre-surgical prostate MRI and histopathology images via super-resolution volume reconstruction[J].Medical Image Analysis, 2021, 47(9): 57-70.
9 Qiu D F, Zheng L X, Zhu J Q, et al.Multiple improved residual networks for medical image super-resolution[J]. Future Generation Computer Systems, 2021, 116(9): 200-208.
10 杨巨成, 左美然, 魏峰, 等.基于误差反馈和面部后先验信息的人脸超分辨率重建[J]. 天津科技大学学报, 2022, 37(2): 35-42.
Yang Ju-cheng, Zuo Mei-ran, Wei Feng, et al.Face Super-resolution using error feedback and facial posterior[J]. Journal of Tianjin University of Science & Technology, 2022, 37(2): 35-42.
11 王宸, 刘剑飞, 郝禄国, 等.一种基于InsightFace算法的课堂人脸识别方法研究[J].南开大学学报:自然科学版, 2022, 55(2): 59-68.
Wang Chen, Liu Jian-fei, Hao Lu-guo, et al.A classroom face recognition method based on insightface algorithm[J]. Acta Scientiarum Naturalium Universitatis Nankaiensis(Natural Science Edition), 2022, 55(2): 59-68.
12 Jin Y, Zhang Y, Cen Y G,et al.Pedestrian detection with super-resolution reconstruction for low-quality image[J].Pattern Recognition, 2021, 115(3):No.107846.
13 王鑫刚, 田军委, 刘雪松, 等. 基于改进Yolov5模型的实时人脸检测算法[J]. 激光与红外, 2023, 53(4): 633-640.
Wang Xin-gang, Tian Jun-wei, Liu Xue-song, et al.Real-time face detection algorithm based on improved Yolov5 model[J]. Laser & Infrared, 2023, 53(4):633-640.
14 王红星, 胡永阳, 邓超. 基于LBP和ELM的人脸识别算法研究与实现[J]. 河南理工大学学报: 自然科学版, 2021, 40(5): 139-145.
Wang Hong-xing, Hu Yong-yang, Deng Chao. Research and implementation of face recognition algorithm based on LBP and ELM[J]. Journal of Henan Polytechnic University(Natural Science), 2021, 40(5): 139-145.
15 Huan H, Li P C, Zou N, et al.End-to-end super-resolution for remote-sensing images using an improved multi-scale residual network[J].Remote Sensing, 2021, 13(4): 666-681.
16 Song Q, Liu H F.Deep gradient prior regularized robust video super-resolution[J]. Electronics, 2021, 10(14): No.1641.
17 Liu H, Qian Y R, Yang G,et al.Research on super-resolution reconstruction of remote sensing images: a comprehensive review[J]. Optical Engineering, 2021, 60(10): 107-122.
18 Li X G, Dong N, Huang J L,et al.A discriminative self-attention cycle GAN for face super-resolution and recognition[J]. IET Image Processing, 2021, 15(11): 2614-2628.
19 杜延松,曹林,杜康宁,等.一种基于先验生成对抗网络的人脸超分辨率重建方法[J].电讯技术, 2023, 63(5):618-625.
Du Yan-song, Cao Lin, Du Kang-ning, et al. A method for face super-resolution reconstruction based on prior generation adversarial network[J]. Telecommunication Engineering, 2023, 63(5): 618-625.
[1] 赖丹晖,罗伟峰,袁旭东,邱子良. 复杂环境下多模态手势关键点特征提取算法[J]. 吉林大学学报(工学版), 2024, 54(8): 2288-2294.
[2] 温晓岳,钱国敏,孔桦桦,缪月洁,王殿海. TrafficPro:一种针对城市信控路网的路段速度预测框架[J]. 吉林大学学报(工学版), 2024, 54(8): 2214-2222.
[3] 郭昕刚,何颖晨,程超. 抗噪声的分步式图像超分辨率重构算法[J]. 吉林大学学报(工学版), 2024, 54(7): 2063-2071.
[4] 张自超,陈建. 基于双目仿鹰眼视觉与超分辨的果园三维点云重建[J]. 吉林大学学报(工学版), 2024, 54(5): 1469-1481.
[5] 吕卫,韩镓泽,褚晶辉,井佩光. 基于多模态自注意力网络的视频记忆度预测[J]. 吉林大学学报(工学版), 2023, 53(4): 1211-1219.
[6] 白琳,刘林军,李轩昂,吴沙,刘汝庆. 基于自监督学习的单目图像深度估计算法[J]. 吉林大学学报(工学版), 2023, 53(4): 1139-1145.
[7] 时小虎,吴佳琦,吴春国,程石,翁小辉,常志勇. 基于残差网络的弯道增强车道线检测方法[J]. 吉林大学学报(工学版), 2023, 53(2): 584-592.
[8] 江晟,王鹏朗,邓志吉,别一鸣. 基于深度学习的交通事故救援图像融合算法[J]. 吉林大学学报(工学版), 2023, 53(12): 3472-3480.
[9] 曾春艳,严康,王志锋,王正辉. 多尺度生成对抗网络下图像压缩感知重建算法[J]. 吉林大学学报(工学版), 2023, 53(10): 2923-2931.
[10] 林乐平,卢增通,欧阳宁. 面向非配合场景的人脸重建及识别方法[J]. 吉林大学学报(工学版), 2022, 52(12): 2941-2946.
[11] 曹洁,何智栋,余萍,王进花. 数据不平衡分布下轴承故障诊断方法[J]. 吉林大学学报(工学版), 2022, 52(11): 2523-2531.
[12] 陈雪云,许韬,黄小巧. 基于条件生成对抗网络的医学细胞图像生成检测方法[J]. 吉林大学学报(工学版), 2021, 51(4): 1414-1419.
[13] 刘元宁,吴迪,朱晓冬,张齐贤,李双双,郭书君,王超. 基于YOLOv3改进的用户界面组件检测算法[J]. 吉林大学学报(工学版), 2021, 51(3): 1026-1033.
[14] 李锦青,周健,底晓强. 基于循环生成对抗网络的学习型光学图像加密方案[J]. 吉林大学学报(工学版), 2021, 51(3): 1060-1066.
[15] 王小玉,胡鑫豪,韩昌林. 基于生成对抗网络的人脸铅笔画算法[J]. 吉林大学学报(工学版), 2021, 51(1): 285-292.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!