PointFlow: 3D Point Cloud Generation with Continuous Normalizing Flows

Abstract

As 3D point clouds become the representation of choice for multiple vision and graphics applications, the ability to synthesize or reconstruct high-resolution, high-fidelity point clouds becomes crucial. Despite the recent success of deep learning models in discriminative tasks of point clouds, generating point clouds remains challenging. This paper proposes a principled probabilistic framework to generate 3D point clouds by modeling them as a distribution of distributions. Specifically, we learn a two-level hierarchy of distributions where the first level is the distribution of shapes and the second level is the distribution of points given a shape. This formulation allows us to both sample shapes and sample an arbitrary number of points from a shape. Our generative model, named PointFlow, learns each level of the distribution with a continuous normalizing flow. The invertibility of normalizing flows enables the computation of the likelihood during training and allows us to train our model in the variational inference framework. Empirically, we demonstrate that PointFlow achieves state-of-the-art performance in point cloud generation. We additionally show that our model can faithfully reconstruct point clouds and learn useful representations in an unsupervised manner. The code will be available at https://github.com/stevenygd/PointFlow.

Published in ICCV 2019

References in corpus (14)

• Auto-Encoding Variational Bayes
• Conditional Image Generation with PixelCNN Decoders
• Pixel Recurrent Neural Networks
• Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modeling
• NICE: Non-linear Independent Components Estimation
• Variational Inference with Normalizing Flows
• Learning Representations and Generative Models for 3D Point Clouds
• Variational Lossy Autoencoder
• An empirical study on evaluation metrics of generative adversarial networks
• Flow-GAN: Combining Maximum Likelihood and Adversarial Learning in Generative Models
• Comparison of Maximum Likelihood and GAN-based training of Real NVPs
• Patch-based Progressive 3D Point Set Upsampling
• PointGrow: Autoregressively Learned Point Cloud Generation with Self-Attention
• PointWise: An Unsupervised Point-wise Feature Learning Network

Cited by in corpus (16)

• Equivariant flow-based sampling for lattice gauge theory
• A Review of Location Encoding for GeoAI: Methods and Applications
• PolyGen: An Autoregressive Generative Model of 3D Meshes
• EPiC-GAN: Equivariant Point Cloud Generation for Particle Jets
• Learning Deformable Tetrahedral Meshes for 3D Reconstruction
• UCSG-Net -- Unsupervised Discovering of Constructive Solid Geometry Tree
• Generating valid Euclidean distance matrices
• Data-Driven Approach to Encoding and Decoding 3-D Crystal Structures
• Cascaded Refinement Network for Point Cloud Completion
• PQ-NET: A Generative Part Seq2Seq Network for 3D Shapes
• Local Deep Implicit Functions for 3D Shape
• TreeFlow: Going beyond Tree-based Gaussian Probabilistic Regression
• Getting Topology and Point Cloud Generation to Mesh
• Energy-Based Processes for Exchangeable Data
• Deep Implicit Templates for 3D Shape Representation
• Learning to Generate 3D Shapes with Generative Cellular Automata