Neural 3D Morphable Models: Spiral Convolutional Networks for 3D Shape Representation Learning and Generation

Abstract

Generative models for 3D geometric data arise in many important applications in 3D computer vision and graphics. In this paper, we focus on 3D deformable shapes that share a common topological structure, such as human faces and bodies. Morphable Models and their variants, despite their linear formulation, have been widely used for shape representation, while most of the recently proposed nonlinear approaches resort to intermediate representations, such as 3D voxel grids or 2D views. In this work, we introduce a novel graph convolutional operator, acting directly on the 3D mesh, that explicitly models the inductive bias of the fixed underlying graph. This is achieved by enforcing consistent local orderings of the vertices of the graph, through the spiral operator, thus breaking the permutation invariance property that is adopted by all the prior work on Graph Neural Networks. Our operator comes by construction with desirable properties (anisotropic, topology-aware, lightweight, easy-to-optimise), and by using it as a building block for traditional deep generative architectures, we demonstrate state-of-the-art results on a variety of 3D shape datasets compared to the linear Morphable Model and other graph convolutional operators.

to appear at ICCV 2019

References in corpus (2)

• Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modeling
• Embodied Hands: Modeling and Capturing Hands and Bodies Together

Cited by in corpus (18)

• Improving Graph Neural Network Expressivity via Subgraph Isomorphism Counting
• Deep Polynomial Neural Networks
• Gauge Equivariant Mesh CNNs: Anisotropic convolutions on geometric graphs
• Single Image 3D Hand Reconstruction with Mesh Convolutions
• Neural Human Deformation Transfer
• SpiralNet++: A Fast and Highly Efficient Mesh Convolution Operator
• BCNet: Learning Body and Cloth Shape from A Single Image
• Projection-based Classification of Surfaces for 3D Human Mesh Sequence Retrieval
• Unsupervised Shape and Pose Disentanglement for 3D Meshes
• Deep Spectral Meshes: Multi-Frequency Facial Mesh Processing with Graph Neural Networks
• TailorMe: Self-Supervised Learning of an Anatomically Constrained Volumetric Human Shape Model
• Learning Feature Aggregation for Deep 3D Morphable Models
• Geometrically Principled Connections in Graph Neural Networks
• Disentangled Face Identity Representations for joint 3D Face Recognition and Expression Neutralisation
• Unsupervised Geometric Disentanglement for Surfaces via CFAN-VAE
• LCollision: Fast Generation of Collision-Free Human Poses using Learned Non-Penetration Constraints
• 3D Shapes Local Geometry Codes Learning with SDF
• Unsupervised Dense Deformation Embedding Network for Template-Free Shape Correspondence