Learning a Hierarchical Latent-Variable Model of 3D Shapes

Abstract

We propose the Variational Shape Learner (VSL), a generative model that learns the underlying structure of voxelized 3D shapes in an unsupervised fashion. Through the use of skip-connections, our model can successfully learn and infer a latent, hierarchical representation of objects. Furthermore, realistic 3D objects can be easily generated by sampling the VSL's latent probabilistic manifold. We show that our generative model can be trained end-to-end from 2D images to perform single image 3D model retrieval. Experiments show, both quantitatively and qualitatively, the improved generalization of our proposed model over a range of tasks, performing better or comparable to various state-of-the-art alternatives.

Accepted as oral presentation at International Conference on 3D Vision (3DV), 2018

References in corpus (10)

• PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation
• Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modeling
• Tutorial on Variational Autoencoders
• DRAW: A Recurrent Neural Network For Image Generation
• Variational Autoencoder for Deep Learning of Images, Labels and Captions
• Towards Principled Methods for Training Generative Adversarial Networks
• Learning a Multi-View Stereo Machine
• Towards a Neural Statistician
• Unsupervised Learning of 3D Structure from Images
• 3D-R2N2: A Unified Approach for Single and Multi-view 3D Object Reconstruction