吉林大学学报(工学版) ›› 2022, Vol. 52 ›› Issue (12): 2971-2983.doi: 10.13229/j.cnki.jdxbgxb20210526

• 计算机科学与技术 • 上一篇    下一篇

基于无监督变形网络的三维模型稠密对应关系计算

杨军1,2(),高志明1   

  1. 1.兰州交通大学 电子与信息工程学院,兰州 730070
    2.兰州交通大学 测绘与地理信息学院,兰州 730070
  • 收稿日期:2021-06-18 出版日期:2022-12-01 发布日期:2022-12-08
  • 作者简介:杨军(1973-),男,教授,博士生导师. 研究方向:三维模型空间分析,遥感影像分析与处理,机器学习. E-mail:yangj@mail.lzjtu.cn
  • 基金资助:
    国家自然科学基金项目(42261067);甘肃省科技计划项目(20JR5RA429);兰州市人才创新创业项目(2020-RC-22);兰州交通大学天佑创新团队项目(TY202002)

Dense correspondence calculation of 3D models based on unsupervised deformed network

Jun YANG1,2(),Zhi-ming GAO1   

  1. 1.School of Electronic and Information Engineering,Lanzhou Jiaotong University,Lanzhou 730070,China
    2.Faculty of Geomatics,Lanzhou Jiaotong University,Lanzhou 730070,China
  • Received:2021-06-18 Online:2022-12-01 Published:2022-12-08

摘要:

针对利用深度学习方法计算三维模型间稠密对应关系时算法泛化能力较差的问题,提出了一种基于无监督变形网络的对应关系的计算方法。首先,通过基于螺旋卷积的模板特征提取网络计算非参数化模板的局部特征描述符;其次,在模板变形网络中利用编码器获取输入模型的全局特征,并与模板的局部特征描述符拼接,将拼接后的特征输入到解码器中计算模板的变形模型;最后,通过最近邻搜索算法计算变形模型与输入模型之间的稠密对应关系。实验结果表明,在具有相同训练样本的条件下,本文算法与目前主流算法相比得到了更精确的稠密对应关系,且具有更强的泛化能力。

关键词: 计算机应用, 对应关系, 模型变形, 几何深度学习, 无监督, 图卷积神经网络

Abstract:

Aiming at the poor generalization ability of dense correspondence computation algorithms of the 3D shapes using deep learning, a method for computing correspondence based on unsupervised deformable networks was proposed. Firstly, the local feature descriptors of non-parametric templates were extracted by the template feature extraction network based on spiral convolution. Secondly, the global features of the input shape were obtained through the encoder in the template deformation network, and spliced with the local feature descriptors of the template. Then the joined features were input into the decoder to calculate the deformation model of the template. Finally, the dense correspondence between the deformation model and the input model was calculated by the nearest neighbor search algorithm. The experiment results show that the proposed algorithm can obtain more accurate dense correspondence under the same training dataset and has a stronger generalization ability than that of the current mainstream algorithms.

Key words: computer application, correspondence, model deformation, geometric deep learning, unsupervised, graph convolutional neural network

中图分类号: 

  • TP391.4

图1

螺旋序列"

图2

模板特征提取网络"

图3

模板变形网络"

图4

倒角距离"

图5

拉普拉斯算子"

图6

FAUST数据集中不同的人体和姿态"

表1

不同算法的运行时间比较"

算法预处理时间/s

训练和测试

时间/s

总运行时间/s
3D-CODED1250005062
DFMNet7015001570
SURFMNet153570
Unsup FMNet154580
本文12198260

图7

在FAUST数据集上本文算法与不同算法构建的对应关系结果对比"

图8

在FAUST数据集上本文算法与不同算法构建的对应关系纹理映射结果对比"

图9

在CoMa数据集上本文算法与不同算法构建的对应关系结果对比"

图10

本文算法在SMAL5K数据集上的测试结果"

图11

测地误差曲线"

表2

不同算法的平均测地误差 (%)"

算法FAUST数据集CoMa数据集
SURFMNet0.320.29
Unsup FMNet0.220.30
DFMNet0.330.54
3D-CODED0.330.26
本文0.150.09
1 Corman T, Ovsjanikov M, Chambolle A. Supervised descriptor learning for non-rigid shape matching[C]∥European Conference on Computer Vision(ECCV), Switzerland, Zurich, 2014: 283-298.
2 Litany O, Remez T, Rodolà E, et al. Deep functional maps: structured prediction for dense shape correspondence[C]∥International Conference on Computer Vision, Venice, Italy, 2017: 5659-5667.
3 Chen Q, Koltun V. Robust non-rigid registration by convex optimization[C]∥International Conference on Computer Vision, Chile, Santiago, 2015: 2039-2047.
4 Van K, Zhang H, Hamarneh G, et al. A survey on shape correspondence[J]. Computer Graphics Forum, 2011, 30(6): 1681-1707.
5 Groueix T, Fisher M, Kim V, et al. 3D-CODED: 3D correspondences by deep deformation[C]∥European Conference on Computer Vision(ECCV), Munich, Germany, 2018: 235-251.
6 Gao L, Yang J, Wu T, et al. SDM-NET: deep generative network for structured deformable mesh[J]. ACM Transactions on Graphics, 2019, 38(6): 1-15.
7 Wu N, Song S, Khosla A, et al. 3D ShapeNets: a deep representation for volumetric shapes[C]∥IEEE Conference on Computer Vision and Pattern Recognition(CVPR), Boston, USA, 2015: 1912-1920.
8 Tan Q, Gao L, Lai Y, et al. Mesh-based autoencoders for localized deformation component analysis[C]∥AAAI Conference on Artificial Intelligence, New Orleans, USA, 2017: 247-263.
9 Gong W, Chen L, Michael B, et al. SpiralNet++: a fast and highly efficient mesh convolution operator[C]∥International Conference on Computer Vision, Seoul, Korea, 2019: 4141-4148.
10 Fukushima K. Neocognitron: a self-organizing neural network model for a mechanism of pattern recognition unaffected by shift in position[J]. Biological Cybernetics, 1980, 36(4): 193-202.
11 Facundo M, Sapiro G. Theoretical and computational framework for isometry invariant recognition of point cloud data[J]. Foundations of Computational Mathematics, 2005, 5(3): 313-347.
12 Alexander M, Michael M, Ron K, et al. Generalized multidimensional scaling: a framework for isometry-invariant partial surface matching[J]. Proceedings of the National Academy of Sciences, 2006, 103(5): 1168-1172.
13 Rusinkiewicz S, Levoy M. Efficient variants of the ICP algorithm[C]∥Proceedings Third International Conference on 3-D Digital Imaging and Modeling, Quebec, Canada, 2001: 145-152.
14 Sun J, Ovsjanikov M, Guibas L. A concise and provably informative multi-scale signature-based on heat diffusion[J]. Computer Graphics Forum, 2009, 28(5): 1383-1392.
15 Aubry M, Schlickewei U, Cremers D. The wave kernel signature: a quantum mechanical approach to shape analysis[C]∥International Conference on Computer Vision, Barcelona, Spain, 2011: 1626-1633.
16 杨军, 闫寒, 王茂正. 融合特征描述符约束的3维等距模型对应关系计算[J]. 中国图象图形学报, 2016, 21(5): 628-635.
Yang Jun, Yan Han, Wang Mao-zheng. Correspondence calculation of 3D isometric model fused with feature descriptor constraints[J]. Journal of Image and Graphics, 2016, 21(5): 628-635.
17 Solomon J, Nguyen A, Butscher A, et al. Soft maps between surfaces[J]. Computer Graphics Forum, 2012, 31(5): 1617-1626.
18 Kim V G, Li W, Mitra N J, et al. Exploring collections of 3D models using fuzzy correspondences[J]. ACM Transactions on Graphics, 2012, 31(4): No.54.
19 Lipman Y, Funkhouser T A. Mobius voting for surface correspondence[J]. ACM Transactions on Graphics, 2009, 28(3):1-12.
20 Ovsjanikov M, Quentin M, Facundo M, et al. One point isometric matching with the heat kernel[J]. Computer Graphics Forum, 2010, 29(5): 1555-1564.
21 Masci J, Boscaini D, Bronstein M, et al. Geodesic convolutional neural networks on Riemannian manifolds[C]∥International Conference on Computer Vision, Santiago, Chile, 2015: 37-45.
22 Ovsjanikov M, Ben-Chen M, Solomon J, et al. Functional maps: a flexible representation of maps between shapes[J]. ACM Transactions on Graphics, 2012, 31(4): 1-11.
23 杨军, 闫寒. 校准三维模型基矩阵的函数映射的对应关系计算[J]. 武汉大学学报: 信息科学版, 2018, 43(10): 1518-1525.
Yang Jun, Yan Han. Correspondence calculation of functional maps for calibrating the base matrix of 3D model[J]. Geomatics and Information Science of Wuhan University, 2018, 43(10): 1518-1525.
24 Wu Y, Yang J, Zhao J. Partial 3D shape functional correspondence via fully spectral eigenvalue alignment and upsampling refinement[J]. Computers & Graphics, 2020, 92: 99-113.
25 Salti S, Tombari F, Di S. Shot: unique signatures of histograms for surface and texture description[J]. Computer Vision and Image Understanding, 2014, 125(8): 251-264.
26 Qi C, Su H, Mo K, et al. PointNet: deep learning on point sets for 3D classification and segmentation[C]∥IEEE Conference on Computer Vision and Pattern Recognition(CVPR), Honolulu, Hawaii, USA, 2017: 652-660.
27 Lu D, Fang Y. Meta deformation network: meta functional for shape correspondence[C]∥IEEE Conference on Computer Vision and Pattern Recognition(CVPR), Seattle, USA, 2020: 6647-6660.
28 Groueix T, Fisher M, Kim V G, et al. Unsupervised cycle-consistent deformation for shape matching[J]. Computer Graphics Forum, 2019, 38(5): 123-133.
29 Vestner M, Litman R, Rodola E, et al. Product manifold filter: non-rigid shape correspondence via kernel density estimation in the product space[C]∥IEEE Conference on Computer Vision and Pattern Recognition(CVPR), Honolulu, Hawaii, 2017: 6681-6690.
30 Federica B, Javier R, Matthew L, et al. Faust: dataset and evaluation for 3D mesh registration[C]∥IEEE Conference on Computer Vision and Pattern Recognition(CVPR), Columbus, USA, 2014: 3794-3801.
31 Varol G, Romero J, Martin X, et al. Learning from synthetic humans[C]∥IEEE Conference on Computer Vision and Pattern Recognition(CVPR), Honolulu, Hawaii, USA, 2017: 109-117.
32 Ranjan A, Bolkart T, Sanyal S, et al. Generating 3D faces using convolutional mesh autoencoders[C]∥European Conference on Computer Vision(ECCV), Munich, Germany, 2018: 704-720.
33 Zuffi S, Kanazawa A, Jacobs D, et al. 3D menagerie: modeling the 3D shape and pose of animals[C]∥IEEE Conference on Computer Vision and Pattern Recognition(CVPR), Honolulu, Hawaii, USA, 2017: 6365-6373.
34 Matthew L, Naureen M, Javier R, et al. Smpl: a skinned multi-person linear model[J]. ACM Transactions on Graphics, 2015, 34(6): 283-298.
35 Lowe D G. Distinctive image features from scale-invariant keypoints[J]. International Journal of Computer Vision, 2004, 60(2): 91-110.
36 Halimi O, Litany O, Rodola E, et al. Unsupervised learning of dense shape correspondence[C]∥IEEE Conference on Computer Vision and Pattern Recognition(CVPR), Long Beach, USA, 2019: 4370-4379.
37 Roufosse J M, Sharma A, Ovsjanikov M. Unsupervised deep learning for structured shape matching[C]∥International Conference on Computer Vision, Seoul, Korea, 2019: 5659-5667.
[1] 祁贤雨,王巍,王琳,赵玉飞,董彦鹏. 基于物体语义栅格地图的语义拓扑地图构建方法[J]. 吉林大学学报(工学版), 2023, 53(2): 569-575.
[2] 时小虎,吴佳琦,吴春国,程石,翁小辉,常志勇. 基于残差网络的弯道增强车道线检测方法[J]. 吉林大学学报(工学版), 2023, 53(2): 584-592.
[3] 郭鹏,赵文超,雷坤. 基于改进Jaya算法的双资源约束柔性作业车间调度[J]. 吉林大学学报(工学版), 2023, 53(2): 480-487.
[4] 刘近贞,高国辉,熊慧. 用于脑组织分割的多尺度注意网络[J]. 吉林大学学报(工学版), 2023, 53(2): 576-583.
[5] 赵宏伟,张健荣,朱隽平,李海. 基于对比自监督学习的图像分类框架[J]. 吉林大学学报(工学版), 2022, 52(8): 1850-1856.
[6] 秦贵和,黄俊锋,孙铭会. 基于双手键盘的虚拟现实文本输入[J]. 吉林大学学报(工学版), 2022, 52(8): 1881-1888.
[7] 胡丹,孟新. 基于时变网格的对地观测卫星搜索海上船舶方法[J]. 吉林大学学报(工学版), 2022, 52(8): 1896-1903.
[8] 曲福恒,丁天雨,陆洋,杨勇,胡雅婷. 基于邻域相似性的图像码字快速搜索算法[J]. 吉林大学学报(工学版), 2022, 52(8): 1865-1871.
[9] 白天,徐明蔚,刘思铭,张佶安,王喆. 基于深度神经网络的诉辩文本争议焦点识别[J]. 吉林大学学报(工学版), 2022, 52(8): 1872-1880.
[10] 周丰丰,朱海洋. 基于三段式特征选择策略的脑电情感识别算法SEE[J]. 吉林大学学报(工学版), 2022, 52(8): 1834-1841.
[11] 周丰丰,张亦弛. 基于稀疏自编码器的无监督特征工程算法BioSAE[J]. 吉林大学学报(工学版), 2022, 52(7): 1645-1656.
[12] 王军,徐彦惠,李莉. 低能耗支持完整性验证的数据融合隐私保护方法[J]. 吉林大学学报(工学版), 2022, 52(7): 1657-1665.
[13] 王生生,姜林延,杨永波. 基于最优传输特征选择的医学图像分割迁移学习[J]. 吉林大学学报(工学版), 2022, 52(7): 1626-1638.
[14] 康耀龙,冯丽露,张景安,陈富. 基于谱聚类的高维类别属性数据流离群点挖掘算法[J]. 吉林大学学报(工学版), 2022, 52(6): 1422-1427.
[15] 王文军,余银峰. 考虑数据稀疏的知识图谱缺失连接自动补全算法[J]. 吉林大学学报(工学版), 2022, 52(6): 1428-1433.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
[1] 朱冰,赵健,李静,李幼德,刘巍,魏青 . 面向牵引力控制系统的AMESim与
MATLAB联合仿真平台
[J]. 吉林大学学报(工学版), 2008, 38(增刊): 23 -0027 .
[2] 马顺利,李明哲,孙刚,李湘吉,钱直睿 . 拼焊板多点成形过程的数值模拟[J]. 吉林大学学报(工学版), 2008, 38(02): 334 -0339 .
[3] 陶敏,李建桥,杨印生,李洪伟,潘燕. 金属基仿生减阻材料元素与性能量化分析[J]. 吉林大学学报(工学版), 2006, 36(03): 367 -0370 .
[4] 王健,葛安林,雷雨龙,田华 . 液力变矩器叶片三维成型法及其性能分析[J]. 吉林大学学报(工学版), 2007, 37(01): 43 -47 .
[5] 李晓磊,李丹,殷涌光. 酶法制备直链麦芽寡糖[J]. 吉林大学学报(工学版), 2009, 39(06): 1559 -1562 .
[6] 宋传学, 高晋, 杨树凯. 悬架对整车平顺性影响的虚拟仿真分析[J]. 吉林大学学报(工学版), 2010, 40(增刊): 6 -00012 .
[7] 郭耸, 顾国昌, 蔡则苏, 刘海波, 沈晶. 基于肤色分割和改进的AdaBoostSVM算法的人脸检测[J]. 吉林大学学报(工学版), 2011, 41(02): 473 -0478 .
[8] 宋雨来,刘耀辉,朱先勇,王素环,于思荣. 钕对AZ91镁合金组织及机械性能的影响[J]. 吉林大学学报(工学版), 2006, 36(03): 289 -0293 .
[9] 鹿应荣,杨印生,吕锋 . 基于模糊聚类分析的车辆优化调度[J]. 吉林大学学报(工学版), 2006, 36(增刊2): 147 -151 .
[10] 张英锋, 马彪, 朱愿, 张金乐. 基于超球面支持向量机的综合传动状态判别[J]. 吉林大学学报(工学版), 2012, 42(01): 13 -18 .