Journal of Jilin University(Engineering and Technology Edition) ›› 2022, Vol. 52 ›› Issue (12): 2971-2983.doi: 10.13229/j.cnki.jdxbgxb20210526

Previous Articles     Next Articles

Dense correspondence calculation of 3D models based on unsupervised deformed network

Jun YANG1,2(),Zhi-ming GAO1   

  1. 1.School of Electronic and Information Engineering，Lanzhou Jiaotong University，Lanzhou 730070，China
    2.Faculty of Geomatics，Lanzhou Jiaotong University，Lanzhou 730070，China
  • Received:2021-06-18 Online:2022-12-01 Published:2022-12-08

RICH HTML

 2   

PDF (PC)

143

Abstract:

Aiming at the poor generalization ability of dense correspondence computation algorithms of the 3D shapes using deep learning， a method for computing correspondence based on unsupervised deformable networks was proposed. Firstly， the local feature descriptors of non-parametric templates were extracted by the template feature extraction network based on spiral convolution. Secondly， the global features of the input shape were obtained through the encoder in the template deformation network， and spliced with the local feature descriptors of the template. Then the joined features were input into the decoder to calculate the deformation model of the template. Finally， the dense correspondence between the deformation model and the input model was calculated by the nearest neighbor search algorithm. The experiment results show that the proposed algorithm can obtain more accurate dense correspondence under the same training dataset and has a stronger generalization ability than that of the current mainstream algorithms.

Key words: computer application, correspondence, model deformation, geometric deep learning, unsupervised, graph convolutional neural network

CLC Number: 

  • TP391.4

Fig.1

Spiral sequence"

Fig.2

Template feature extraction network"

Fig.3

Template deformation network"

Fig.4

Chamfer distance"

Fig.5

Laplacian operator"

Fig.6

Different human body and posturein the FAUST dataset"

Table 1

Comparison of running time of different algorithms"

算法预处理时间/s

训练和测试

时间/s

总运行时间/s
3D-CODED1250005062
DFMNet7015001570
SURFMNet153570
Unsup FMNet154580
本文12198260

Fig.7

Comparison between our method and other methods for constructing correspondence on FAUST dataset"

Fig.8

Comparison between our method and other methods for texture transfer on FAUST dataset"

Fig.9

Comparison between our method and other methods for constructing correspondence on CoMa dataset"

Fig.10

Test results of our algorithm on the SMAL5K dataset"

Fig.11

Geodetic error curve"

Table 2

Average geodetic error of different algorithms"

算法FAUST数据集CoMa数据集
SURFMNet0.320.29
Unsup FMNet0.220.30
DFMNet0.330.54
3D-CODED0.330.26
本文0.150.09
1 Corman T, Ovsjanikov M, Chambolle A. Supervised descriptor learning for non-rigid shape matching[C]∥European Conference on Computer Vision(ECCV), Switzerland, Zurich, 2014: 283-298.
2 Litany O, Remez T, Rodolà E, et al. Deep functional maps: structured prediction for dense shape correspondence[C]∥International Conference on Computer Vision, Venice, Italy, 2017: 5659-5667.
3 Chen Q, Koltun V. Robust non-rigid registration by convex optimization[C]∥International Conference on Computer Vision, Chile, Santiago, 2015: 2039-2047.
4 Van K, Zhang H, Hamarneh G, et al. A survey on shape correspondence[J]. Computer Graphics Forum, 2011, 30(6): 1681-1707.
5 Groueix T, Fisher M, Kim V, et al. 3D-CODED: 3D correspondences by deep deformation[C]∥European Conference on Computer Vision(ECCV), Munich, Germany, 2018: 235-251.
6 Gao L, Yang J, Wu T, et al. SDM-NET: deep generative network for structured deformable mesh[J]. ACM Transactions on Graphics, 2019, 38(6): 1-15.
7 Wu N, Song S, Khosla A, et al. 3D ShapeNets: a deep representation for volumetric shapes[C]∥IEEE Conference on Computer Vision and Pattern Recognition(CVPR), Boston, USA, 2015: 1912-1920.
8 Tan Q, Gao L, Lai Y, et al. Mesh-based autoencoders for localized deformation component analysis[C]∥AAAI Conference on Artificial Intelligence, New Orleans, USA, 2017: 247-263.
9 Gong W, Chen L, Michael B, et al. SpiralNet++: a fast and highly efficient mesh convolution operator[C]∥International Conference on Computer Vision, Seoul, Korea, 2019: 4141-4148.
10 Fukushima K. Neocognitron: a self-organizing neural network model for a mechanism of pattern recognition unaffected by shift in position[J]. Biological Cybernetics, 1980, 36(4): 193-202.
11 Facundo M, Sapiro G. Theoretical and computational framework for isometry invariant recognition of point cloud data[J]. Foundations of Computational Mathematics, 2005, 5(3): 313-347.
12 Alexander M, Michael M, Ron K, et al. Generalized multidimensional scaling: a framework for isometry-invariant partial surface matching[J]. Proceedings of the National Academy of Sciences, 2006, 103(5): 1168-1172.
13 Rusinkiewicz S, Levoy M. Efficient variants of the ICP algorithm[C]∥Proceedings Third International Conference on 3-D Digital Imaging and Modeling, Quebec, Canada, 2001: 145-152.
14 Sun J, Ovsjanikov M, Guibas L. A concise and provably informative multi-scale signature-based on heat diffusion[J]. Computer Graphics Forum, 2009, 28(5): 1383-1392.
15 Aubry M, Schlickewei U, Cremers D. The wave kernel signature: a quantum mechanical approach to shape analysis[C]∥International Conference on Computer Vision, Barcelona, Spain, 2011: 1626-1633.
16 杨军, 闫寒, 王茂正. 融合特征描述符约束的3维等距模型对应关系计算[J]. 中国图象图形学报, 2016, 21(5): 628-635.
Yang Jun, Yan Han, Wang Mao-zheng. Correspondence calculation of 3D isometric model fused with feature descriptor constraints[J]. Journal of Image and Graphics, 2016, 21(5): 628-635.
17 Solomon J, Nguyen A, Butscher A, et al. Soft maps between surfaces[J]. Computer Graphics Forum, 2012, 31(5): 1617-1626.
18 Kim V G, Li W, Mitra N J, et al. Exploring collections of 3D models using fuzzy correspondences[J]. ACM Transactions on Graphics, 2012, 31(4): No.54.
19 Lipman Y, Funkhouser T A. Mobius voting for surface correspondence[J]. ACM Transactions on Graphics, 2009, 28(3):1-12.
20 Ovsjanikov M, Quentin M, Facundo M, et al. One point isometric matching with the heat kernel[J]. Computer Graphics Forum, 2010, 29(5): 1555-1564.
21 Masci J, Boscaini D, Bronstein M, et al. Geodesic convolutional neural networks on Riemannian manifolds[C]∥International Conference on Computer Vision, Santiago, Chile, 2015: 37-45.
22 Ovsjanikov M, Ben-Chen M, Solomon J, et al. Functional maps: a flexible representation of maps between shapes[J]. ACM Transactions on Graphics, 2012, 31(4): 1-11.
23 杨军, 闫寒. 校准三维模型基矩阵的函数映射的对应关系计算[J]. 武汉大学学报: 信息科学版, 2018, 43(10): 1518-1525.
Yang Jun, Yan Han. Correspondence calculation of functional maps for calibrating the base matrix of 3D model[J]. Geomatics and Information Science of Wuhan University, 2018, 43(10): 1518-1525.
24 Wu Y, Yang J, Zhao J. Partial 3D shape functional correspondence via fully spectral eigenvalue alignment and upsampling refinement[J]. Computers & Graphics, 2020, 92: 99-113.
25 Salti S, Tombari F, Di S. Shot: unique signatures of histograms for surface and texture description[J]. Computer Vision and Image Understanding, 2014, 125(8): 251-264.
26 Qi C, Su H, Mo K, et al. PointNet: deep learning on point sets for 3D classification and segmentation[C]∥IEEE Conference on Computer Vision and Pattern Recognition(CVPR), Honolulu, Hawaii, USA, 2017: 652-660.
27 Lu D, Fang Y. Meta deformation network: meta functional for shape correspondence[C]∥IEEE Conference on Computer Vision and Pattern Recognition(CVPR), Seattle, USA, 2020: 6647-6660.
28 Groueix T, Fisher M, Kim V G, et al. Unsupervised cycle-consistent deformation for shape matching[J]. Computer Graphics Forum, 2019, 38(5): 123-133.
29 Vestner M, Litman R, Rodola E, et al. Product manifold filter: non-rigid shape correspondence via kernel density estimation in the product space[C]∥IEEE Conference on Computer Vision and Pattern Recognition(CVPR), Honolulu, Hawaii, 2017: 6681-6690.
30 Federica B, Javier R, Matthew L, et al. Faust: dataset and evaluation for 3D mesh registration[C]∥IEEE Conference on Computer Vision and Pattern Recognition(CVPR), Columbus, USA, 2014: 3794-3801.
31 Varol G, Romero J, Martin X, et al. Learning from synthetic humans[C]∥IEEE Conference on Computer Vision and Pattern Recognition(CVPR), Honolulu, Hawaii, USA, 2017: 109-117.
32 Ranjan A, Bolkart T, Sanyal S, et al. Generating 3D faces using convolutional mesh autoencoders[C]∥European Conference on Computer Vision(ECCV), Munich, Germany, 2018: 704-720.
33 Zuffi S, Kanazawa A, Jacobs D, et al. 3D menagerie: modeling the 3D shape and pose of animals[C]∥IEEE Conference on Computer Vision and Pattern Recognition(CVPR), Honolulu, Hawaii, USA, 2017: 6365-6373.
34 Matthew L, Naureen M, Javier R, et al. Smpl: a skinned multi-person linear model[J]. ACM Transactions on Graphics, 2015, 34(6): 283-298.
35 Lowe D G. Distinctive image features from scale-invariant keypoints[J]. International Journal of Computer Vision, 2004, 60(2): 91-110.
36 Halimi O, Litany O, Rodola E, et al. Unsupervised learning of dense shape correspondence[C]∥IEEE Conference on Computer Vision and Pattern Recognition(CVPR), Long Beach, USA, 2019: 4370-4379.
37 Roufosse J M, Sharma A, Ovsjanikov M. Unsupervised deep learning for structured shape matching[C]∥International Conference on Computer Vision, Seoul, Korea, 2019: 5659-5667.
[1] Xian-yu QI,Wei WANG,Lin WANG,Yu-fei ZHAO,Yan-peng DONG. Semantic topological map building with object semantic grid map [J]. Journal of Jilin University(Engineering and Technology Edition), 2023, 53(2): 569-575.
[2] Xiao-hu SHI,Jia-qi WU,Chun-guo WU,Shi CHENG,Xiao-hui WENG,Zhi-yong CHANG. Residual network based curve enhanced lane detection method [J]. Journal of Jilin University(Engineering and Technology Edition), 2023, 53(2): 584-592.
[3] Peng GUO,Wen-chao ZHAO,Kun LEI. Dual⁃resource constrained flexible job shop optimal scheduling based on an improved Jaya algorithm [J]. Journal of Jilin University(Engineering and Technology Edition), 2023, 53(2): 480-487.
[4] Jin-Zhen Liu,Guo-Hui Gao,Hui Xiong. Multi⁃scale attention network for brain tissue segmentation [J]. Journal of Jilin University(Engineering and Technology Edition), 2023, 53(2): 576-583.
[5] Gui-he QIN,Jun-feng HUANG,Ming-hui SUN. Text input based on two⁃handed keyboard in virtual environment [J]. Journal of Jilin University(Engineering and Technology Edition), 2022, 52(8): 1881-1888.
[6] Fu-heng QU,Tian-yu DING,Yang LU,Yong YANG,Ya-ting HU. Fast image codeword search algorithm based on neighborhood similarity [J]. Journal of Jilin University(Engineering and Technology Edition), 2022, 52(8): 1865-1871.
[7] Tian BAI,Ming-wei XU,Si-ming LIU,Ji-an ZHANG,Zhe WANG. Dispute focus identification of pleading text based on deep neural network [J]. Journal of Jilin University(Engineering and Technology Edition), 2022, 52(8): 1872-1880.
[8] Feng-feng ZHOU,Hai-yang ZHU. SEE： sense EEG⁃based emotion algorithm via three⁃step feature selection strategy [J]. Journal of Jilin University(Engineering and Technology Edition), 2022, 52(8): 1834-1841.
[9] Feng-feng ZHOU,Yi-chi ZHANG. Unsupervised feature engineering algorithm BioSAE based on sparse autoencoder [J]. Journal of Jilin University(Engineering and Technology Edition), 2022, 52(7): 1645-1656.
[10] Jun WANG,Yan-hui XU,Li LI. Data fusion privacy protection method with low energy consumption and integrity verification [J]. Journal of Jilin University(Engineering and Technology Edition), 2022, 52(7): 1657-1665.
[11] Sheng-sheng WANG,Lin-yan JIANG,Yong-bo YANG. Transfer learning of medical image segmentation based on optimal transport feature selection [J]. Journal of Jilin University(Engineering and Technology Edition), 2022, 52(7): 1626-1638.
[12] Yao-long KANG,Li-lu FENG,Jing-an ZHANG,Fu CHEN. Outlier mining algorithm for high dimensional categorical data streams based on spectral clustering [J]. Journal of Jilin University(Engineering and Technology Edition), 2022, 52(6): 1422-1427.
[13] Wen-jun WANG,Yin-feng YU. Automatic completion algorithm for missing links in nowledge graph considering data sparsity [J]. Journal of Jilin University(Engineering and Technology Edition), 2022, 52(6): 1428-1433.
[14] Xue-yun CHEN,Xue-yu BEI,Qu YAO,Xin JIN. Pedestrian segmentation and detection in multi-scene based on G-UNet [J]. Journal of Jilin University(Engineering and Technology Edition), 2022, 52(4): 925-933.
[15] Shi-min FANG. Multiple source data selective integration algorithm based on frequent pattern tree [J]. Journal of Jilin University(Engineering and Technology Edition), 2022, 52(4): 885-890.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
[1] Zhu Bing, Zhao Jian, Li Jing, Li You-de, Liu Wei, Wei Qing . Co-simulation platform with AMESim and MATLAB for traction control system[J]. 吉林大学学报(工学版), 2008, 38(增刊): 23 -0027 .
[2] Ma Shun-li,Li Ming-zhe,Sun Gang,Li Xiang-ji,Qian Zhi-rui . Numerical simulation on multipoint forming
process for tailorwelded blank[J]. 吉林大学学报(工学版), 2008, 38(02): 334 -0339 .
[3] Tao Min, Li Jian-qiao, Yang Yin-sheng, Li Hong-wei, Pan Yan. Quantitative method for relationship between elements and properties of metalbased bionic drag reduction materials[J]. 吉林大学学报(工学版), 2006, 36(03): 367 -0370 .
[4] Wang Jian, Ge An-lin,Lei Yu-long,Tian Hua . Three-dimensional design for hydraulic torque converter blades and its performance analysis[J]. 吉林大学学报(工学版), 2007, 37(01): 43 -47 .
[5] LI Xiao-lei1,LI Dan,YIN Yong-guang. Enzymatic preparation of linear malto oligosaccharides[J]. 吉林大学学报(工学版), 2009, 39(06): 1559 -1562 .
[6] SONG Chuan-Xue, GAO Jin, YANG Shu-Kai. Virtual simulation analysis about influence of suspension on full vehicle ride comfort[J]. 吉林大学学报(工学版), 2010, 40(增刊): 6 -00012 .
[7] GUO Song, GU Guo-Chang, CA Ze-Su, LIU Hai-Bo, SHEN Jing. Face detection based on skin color segmentation and improved AdaBoostSVM algorithm[J]. 吉林大学学报(工学版), 2011, 41(02): 473 -0478 .
[8] Song Yu-lai, Liu Yao-hui, Zhu Xian-yong, Wang Su-huan, Yu Si-rong. Effect of neodymium on microstructure and mechanical properties of AZ91 magnesium alloy[J]. 吉林大学学报(工学版), 2006, 36(03): 289 -0293 .
[9] Lu Ying-rong，Yang Yin-sheng，Lv Feng . Optimal vehicle routing problem based on fuzzy clustering analysis[J]. 吉林大学学报(工学版), 2006, 36(增刊2): 147 -151 .
[10] ZHANG Ying-feng, MA Biao, ZHU Yuan, ZHANG Jin-le. State identification of power-shift steering transmission based on hypersphere support vector machine[J]. 吉林大学学报(工学版), 2012, 42(01): 13 -18 .
Copyright © Journal of Jilin University(Engineering and Technology Edition), All Rights Reserved.
Tel: +86-431-85095297　Fax: +86-431-85094074　E-mail: xbgxb@jlu.edu.cn
Powered by Beijing Magtech Co. Ltd