Disentangling Geometric Deformation Spaces in Generative Latent Shape Models

Abstract

A complete representation of 3D objects requires characterizing the space of deformations in an interpretable manner, from articulations of a single instance to changes in shape across categories. In this work, we improve on a prior generative model of geometric disentanglement for 3D shapes, wherein the space of object geometry is factorized into rigid orientation, non-rigid pose, and intrinsic shape. The resulting model can be trained from raw 3D shapes, without correspondences, labels, or even rigid alignment, using a combination of classical spectral geometry and probabilistic disentanglement of a structured latent representation space. Our improvements include more sophisticated handling of rotational invariance and the use of a diffeomorphic flow network to bridge latent and spectral space. The geometric structuring of the latent space imparts an interpretable characterization of the deformation space of an object. Furthermore, it enables tasks like pose transfer and pose-aware retrieval without requiring supervision. We evaluate our model on its generative modelling, representation learning, and disentanglement performance, showing improved rotation invariance and intrinsic-extrinsic factorization quality over the prior model.

arXiv commentAccepted to IJCV

References in corpus (15)

• Adam: A Method for Stochastic Optimization
• PyTorch: An Imperative Style, High-Performance Deep Learning Library
• PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation
• Normalizing Flows: An Introduction and Review of Current Methods
• Learning from Synthetic Humans
• Normalizing Flows for Probabilistic Modeling and Inference
• From Variational to Deterministic Autoencoders
• LIMP: Learning Latent Shape Representations with Metric Preservation Priors
• Isospectralization, or how to hear shape, style, and correspondence
• Pointwise Rotation-Invariant Network with Adaptive Sampling and 3D Spherical Voxel Convolution
• Towards Spectral Geometric Methods for Euclidean Quantum Gravity
• Rotation Invariant Convolutions for 3D Point Clouds Deep Learning
• Latent feature disentanglement for 3D meshes
• Steklov Spectral Geometry for Extrinsic Shape Analysis
• Learning disentangled representations via product manifold projection

Cited by in corpus (2)

• Neural Human Deformation Transfer
• 3D Generative Model Latent Disentanglement via Local Eigenprojection