ChartPointFlow for Topology-Aware 3D Point Cloud Generation

Abstract

A point cloud serves as a representation of the surface of a three-dimensional (3D) shape. Deep generative models have been adapted to model their variations typically using a map from a ball-like set of latent variables. However, previous approaches did not pay much attention to the topological structure of a point cloud, despite that a continuous map cannot express the varying numbers of holes and intersections. Moreover, a point cloud is often composed of multiple subparts, and it is also difficult to express. In this study, we propose ChartPointFlow, a flow-based generative model with multiple latent labels for 3D point clouds. Each label is assigned to points in an unsupervised manner. Then, a map conditioned on a label is assigned to a continuous subset of a point cloud, similar to a chart of a manifold. This enables our proposed model to preserve the topological structure with clear boundaries, whereas previous approaches tend to generate blurry point clouds and fail to generate holes. The experimental results demonstrate that ChartPointFlow achieves state-of-the-art performance in terms of generation and reconstruction compared with other point cloud generators. Moreover, ChartPointFlow divides an object into semantic subparts using charts, and it demonstrates superior performance in case of unsupervised segmentation.

arXiv commentAccepted to ACM International Conference on Multimedia (ACMMM2021) as an oral presentation

References in corpus (24)

• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• Auto-Encoding Variational Bayes
• Conditional Generative Adversarial Nets
• Categorical Reparameterization with Gumbel-Softmax
• Semi-Supervised Learning with Deep Generative Models
• Variational Inference with Normalizing Flows
• Density estimation using Real NVP
• Learning Representations and Generative Models for 3D Point Clouds
• Masked Autoregressive Flow for Density Estimation
• Glow: Generative Flow with Invertible 1x1 Convolutions
• Towards a Neural Statistician
• Differentiable Manifold Reconstruction for Point Cloud Denoising
• Normalizing Flows on Tori and Spheres
• L2G Auto-encoder: Understanding Point Clouds by Local-to-Global Reconstruction with Hierarchical Self-Attention
• Revisiting Classifier Two-Sample Tests
• i-RevNet: Deep Invertible Networks
• SoftFlow: Probabilistic Framework for Normalizing Flow on Manifolds
• Deep Learning for 3D Point Clouds: A Survey
• SurVAE Flows: Surjections to Bridge the Gap between VAEs and Flows
• Multiresolution Tree Networks for 3D Point Cloud Processing
• Neural Manifold Ordinary Differential Equations
• Learning Gradient Fields for Shape Generation
• Progressive Point Cloud Deconvolution Generation Network
• Spectral-GANs for High-Resolution 3D Point-cloud Generation