Learning 3D Shape Completion under Weak Supervision

Abstract

We address the problem of 3D shape completion from sparse and noisy point clouds, a fundamental problem in computer vision and robotics. Recent approaches are either data-driven or learning-based: Data-driven approaches rely on a shape model whose parameters are optimized to fit the observations; Learning-based approaches, in contrast, avoid the expensive optimization step by learning to directly predict complete shapes from incomplete observations in a fully-supervised setting. However, full supervision is often not available in practice. In this work, we propose a weakly-supervised learning-based approach to 3D shape completion which neither requires slow optimization nor direct supervision. While we also learn a shape prior on synthetic data, we amortize, i.e., learn, maximum likelihood fitting using deep neural networks resulting in efficient shape completion without sacrificing accuracy. On synthetic benchmarks based on ShapeNet and ModelNet as well as on real robotics data from KITTI and Kinect, we demonstrate that the proposed amortized maximum likelihood approach is able to compete with recent fully supervised baselines and outperforms data-driven approaches, while requiring less supervision and being significantly faster.

References in corpus (9)

• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation
• PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space
• Depth Map Prediction from a Single Image using a Multi-Scale Deep Network
• Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modeling
• Perspective Transformer Nets: Learning Single-View 3D Object Reconstruction without 3D Supervision
• Dense 3D Object Reconstruction from a Single Depth View
• Single Image 3D Interpreter Network
• SAMP: Shape and Motion Priors for 4D Vehicle Reconstruction

Cited by in corpus (6)

• Triangle-Net: Towards Robustness in Point Cloud Learning
• DeepJoin: Learning a Joint Occupancy, Signed Distance, and Normal Field Function for Shape Repair
• Learning Pose-invariant 3D Object Reconstruction from Single-view Images
• Combining Shape Completion and Grasp Prediction for Fast and Versatile Grasping with a Multi-Fingered Hand
• An Efficient Deep Learning Approach Using Improved Generative Adversarial Networks for Incomplete Information Completion of Self-driving
• CAOA -- Completion-Assisted Object-CAD Alignment