PointGrow: Autoregressively Learned Point Cloud Generation with Self-Attention

Abstract

Generating 3D point clouds is challenging yet highly desired. This work presents a novel autoregressive model, PointGrow, which can generate diverse and realistic point cloud samples from scratch or conditioned on semantic contexts. This model operates recurrently, with each point sampled according to a conditional distribution given its previously-generated points, allowing inter-point correlations to be well-exploited and 3D shape generative processes to be better interpreted. Since point cloud object shapes are typically encoded by long-range dependencies, we augment our model with dedicated self-attention modules to capture such relations. Extensive evaluations show that PointGrow achieves satisfying performance on both unconditional and conditional point cloud generation tasks, with respect to realism and diversity. Several important applications, such as unsupervised feature learning and shape arithmetic operations, are also demonstrated.

References in corpus (29)

• GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium
• WaveNet: A Generative Model for Raw Audio
• Spectral Networks and Locally Connected Networks on Graphs
• Self-Attention Generative Adversarial Networks
• Conditional Image Generation with PixelCNN Decoders
• Pixel Recurrent Neural Networks
• Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modeling
• Learning Representations and Generative Models for 3D Point Clouds
• Generative and Discriminative Voxel Modeling with Convolutional Neural Networks
• Dynamic Graph CNN for Learning on Point Clouds
• Long Short-Term Memory-Networks for Machine Reading
• Learning shape correspondence with anisotropic convolutional neural networks
• Pixel2Mesh: Generating 3D Mesh Models from Single RGB Images
• PointCNN: Convolution On $\mathcal{X}$-Transformed Points
• RGCNN: Regularized Graph CNN for Point Cloud Segmentation
• SPLATNet: Sparse Lattice Networks for Point Cloud Processing
• A LiDAR Point Cloud Generator: from a Virtual World to Autonomous Driving
• Mining Point Cloud Local Structures by Kernel Correlation and Graph Pooling
• Multiresolution Tree Networks for 3D Point Cloud Processing
• PCN: Point Completion Network
• PVNet: A Joint Convolutional Network of Point Cloud and Multi-View for 3D Shape Recognition
• Im2Avatar: Colorful 3D Reconstruction from a Single Image
• Learning Category-Specific Mesh Reconstruction from Image Collections
• 3D Object Reconstruction from a Single Depth View with Adversarial Learning
• Compact Model Representation for 3D Reconstruction
• Fast Generation for Convolutional Autoregressive Models
• X-Vision: An augmented vision tool with real-time sensing ability in tagged environments
• Variational Autoencoders for Deforming 3D Mesh Models
• Shape Generation using Spatially Partitioned Point Clouds

Cited by in corpus (9)

• Image-based 3D Object Reconstruction: State-of-the-Art and Trends in the Deep Learning Era
• PointFlow: 3D Point Cloud Generation with Continuous Normalizing Flows
• Geometric Back-projection Network for Point Cloud Classification
• Progressive Point Cloud Deconvolution Generation Network
• Self-supervised Feature Learning by Cross-modality and Cross-view Correspondences
• Hypernetwork approach to generating point clouds
• Tranquil Clouds: Neural Networks for Learning Temporally Coherent Features in Point Clouds
• Energy-Based Processes for Exchangeable Data
• Hierarchical View Predictor: Unsupervised 3D Global Feature Learning through Hierarchical Prediction among Unordered Views