Convolutional Occupancy Networks

Abstract

Recently, implicit neural representations have gained popularity for learning-based 3D reconstruction. While demonstrating promising results, most implicit approaches are limited to comparably simple geometry of single objects and do not scale to more complicated or large-scale scenes. The key limiting factor of implicit methods is their simple fully-connected network architecture which does not allow for integrating local information in the observations or incorporating inductive biases such as translational equivariance. In this paper, we propose Convolutional Occupancy Networks, a more flexible implicit representation for detailed reconstruction of objects and 3D scenes. By combining convolutional encoders with implicit occupancy decoders, our model incorporates inductive biases, enabling structured reasoning in 3D space. We investigate the effectiveness of the proposed representation by reconstructing complex geometry from noisy point clouds and low-resolution voxel representations. We empirically find that our method enables the fine-grained implicit 3D reconstruction of single objects, scales to large indoor scenes, and generalizes well from synthetic to real data.

arXiv commentECCV 2020 (Spotlight). Project page with supplementary material and code: https://pengsongyou.github.io/conv_onet

References in corpus (8)

• PyTorch: An Imperative Style, High-Performance Deep Learning Library
• Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modeling
• ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes
• Matterport3D: Learning from RGB-D Data in Indoor Environments
• Learning Efficient Point Cloud Generation for Dense 3D Object Reconstruction
• PIFu: Pixel-Aligned Implicit Function for High-Resolution Clothed Human Digitization
• Learning to Infer Implicit Surfaces without 3D Supervision
• Shape Completion using 3D-Encoder-Predictor CNNs and Shape Synthesis

Cited by in corpus (6)

• Neural Human Performer: Learning Generalizable Radiance Fields for Human Performance Rendering
• Learning Deformable Tetrahedral Meshes for 3D Reconstruction
• UltraSR: Spatial Encoding is a Missing Key for Implicit Image Function-based Arbitrary-Scale Super-Resolution
• NeRV: Neural Representations for Videos
• SPSG: Self-Supervised Photometric Scene Generation from RGB-D Scans
• Neural Ray-Tracing: Learning Surfaces and Reflectance for Relighting and View Synthesis