Generative and Discriminative Voxel Modeling with Convolutional Neural Networks

Abstract

When working with three-dimensional data, choice of representation is key. We explore voxel-based models, and present evidence for the viability of voxellated representations in applications including shape modeling and object classification. Our key contributions are methods for training voxel-based variational autoencoders, a user interface for exploring the latent space learned by the autoencoder, and a deep convolutional neural network architecture for object classification. We address challenges unique to voxel-based representations, and empirically evaluate our models on the ModelNet benchmark, where we demonstrate a 51.5% relative improvement in the state of the art for object classification.

9 pages, 5 figures, 2 tables

Cited by in corpus (22)

• SMASH: One-Shot Model Architecture Search through HyperNetworks
• 3D Point Cloud Classification and Segmentation using 3D Modified Fisher Vector Representation for Convolutional Neural Networks
• A-CNN: Annularly Convolutional Neural Networks on Point Clouds
• Inverse Graphics GAN: Learning to Generate 3D Shapes from Unstructured 2D Data
• 3D Object Reconstruction from a Single Depth View with Adversarial Learning
• Shape Inpainting using 3D Generative Adversarial Network and Recurrent Convolutional Networks
• Weakly Supervised Semantic Segmentation in 3D Graph-Structured Point Clouds of Wild Scenes
• Interactive 3D Modeling with a Generative Adversarial Network
• 3D Shape Synthesis for Conceptual Design and Optimization Using Variational Autoencoders
• A Skeleton-bridged Deep Learning Approach for Generating Meshes of Complex Topologies from Single RGB Images
• Convolutional Neural Networks on non-uniform geometrical signals using Euclidean spectral transformation
• ShapeAdv: Generating Shape-Aware Adversarial 3D Point Clouds
• Geometric features for voxel-based surface recognition
• LP-3DCNN: Unveiling Local Phase in 3D Convolutional Neural Networks
• Discrete Rotation Equivariance for Point Cloud Recognition
• IPC-Net: 3D point-cloud segmentation using deep inter-point convolutional layers
• 3D Object Classification via Spherical Projections
• EnzyNet: enzyme classification using 3D convolutional neural networks on spatial representation
• Volumetric Convolution: Automatic Representation Learning in Unit Ball
• Permutation Matters: Anisotropic Convolutional Layer for Learning on Point Clouds
• libmolgrid: GPU Accelerated Molecular Gridding for Deep Learning Applications
• RocNet: Recursive Octree Network for Efficient 3D Deep Representation