SurfNet: Generating 3D shape surfaces using deep residual networks

Abstract

3D shape models are naturally parameterized using vertices and faces, \ie, composed of polygons forming a surface. However, current 3D learning paradigms for predictive and generative tasks using convolutional neural networks focus on a voxelized representation of the object. Lifting convolution operators from the traditional 2D to 3D results in high computational overhead with little additional benefit as most of the geometry information is contained on the surface boundary. Here we study the problem of directly generating the 3D shape surface of rigid and non-rigid shapes using deep convolutional neural networks. We develop a procedure to create consistent `geometry images' representing the shape surface of a category of 3D objects. We then use this consistent representation for category-specific shape surface generation from a parametric representation or an image by developing novel extensions of deep residual networks for the task of geometry image generation. Our experiments indicate that our network learns a meaningful representation of shape surfaces allowing it to interpolate between shape orientations and poses, invent new shape surfaces and reconstruct 3D shape surfaces from previously unseen images.

CVPR 2017 paper

References in corpus (5)

• Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks
• Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modeling
• FPNN: Field Probing Neural Networks for 3D Data
• Render for CNN: Viewpoint Estimation in Images Using CNNs Trained with Rendered 3D Model Views
• Unsupervised Learning of 3D Structure from Images

Cited by in corpus (10)

• DISN: Deep Implicit Surface Network for High-quality Single-view 3D Reconstruction
• Pixel2Mesh: Generating 3D Mesh Models from Single RGB Images
• Learning Category-Specific Mesh Reconstruction from Image Collections
• Adversarial Generation of Continuous Implicit Shape Representations
• Multi-feature super-resolution network for cloth wrinkle synthesis
• DeepHPS: End-to-end Estimation of 3D Hand Pose and Shape by Learning from Synthetic Depth
• Mesh-based Autoencoders for Localized Deformation Component Analysis
• Sparse Photometric 3D Face Reconstruction Guided by Morphable Models
• LBS Autoencoder: Self-supervised Fitting of Articulated Meshes to Point Clouds
• 3D Surface Reconstruction by Pointillism