Learning Mesh Representations via Binary Space Partitioning Tree Networks

Abstract

Polygonal meshes are ubiquitous, but have only played a relatively minor role in the deep learning revolution. State-of-the-art neural generative models for 3D shapes learn implicit functions and generate meshes via expensive iso-surfacing. We overcome these challenges by employing a classical spatial data structure from computer graphics, Binary Space Partitioning (BSP), to facilitate 3D learning. The core operation of BSP involves recursive subdivision of 3D space to obtain convex sets. By exploiting this property, we devise BSP-Net, a network that learns to represent a 3D shape via convex decomposition without supervision. The network is trained to reconstruct a shape using a set of convexes obtained from a BSP-tree built over a set of planes, where the planes and convexes are both defined by learned network weights. BSP-Net directly outputs polygonal meshes from the inferred convexes. The generated meshes are watertight, compact (i.e., low-poly), and well suited to represent sharp geometry. We show that the reconstruction quality by BSP-Net is competitive with those from state-of-the-art methods while using much fewer primitives. We also explore variations to BSP-Net including using a more generic decoder for reconstruction, more general primitives than planes, as well as training a generative model with variational auto-encoders. Code is available at https://github.com/czq142857/BSP-NET-original.

Accepted to TPAMI. This is the extended journal version of BSP-Net (arXiv:1911.06971) from CVPR 2020

References in corpus (19)

• Auto-Encoding Variational Bayes
• ShapeNet: An Information-Rich 3D Model Repository
• Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modeling
• Binarized Neural Networks
• O-CNN: Octree-based Convolutional Neural Networks for 3D Shape Analysis
• MeshCNN: A Network with an Edge
• Learning Representations and Generative Models for 3D Point Clouds
• Modulating early visual processing by language
• DISN: Deep Implicit Surface Network for High-quality Single-view 3D Reconstruction
• Learning Efficient Point Cloud Generation for Dense 3D Object Reconstruction
• Adaptive O-CNN: A Patch-based Deep Representation of 3D Shapes
• P2P-NET: Bidirectional Point Displacement Net for Shape Transform
• SAGNet:Structure-aware Generative Network for 3D-Shape Modeling
• Multi-chart Generative Surface Modeling
• StructureNet: Hierarchical Graph Networks for 3D Shape Generation
• Multiresolution Tree Networks for 3D Point Cloud Processing
• LOGAN: Unpaired Shape Transform in Latent Overcomplete Space
• SDM-NET: Deep Generative Network for Structured Deformable Mesh
• SCORES: Shape Composition with Recursive Substructure Priors