FoldingNet: Point Cloud Auto-encoder via Deep Grid Deformation

Abstract

Recent deep networks that directly handle points in a point set, e.g., PointNet, have been state-of-the-art for supervised learning tasks on point clouds such as classification and segmentation. In this work, a novel end-to-end deep auto-encoder is proposed to address unsupervised learning challenges on point clouds. On the encoder side, a graph-based enhancement is enforced to promote local structures on top of PointNet. Then, a novel folding-based decoder deforms a canonical 2D grid onto the underlying 3D object surface of a point cloud, achieving low reconstruction errors even for objects with delicate structures. The proposed decoder only uses about 7% parameters of a decoder with fully-connected neural networks, yet leads to a more discriminative representation that achieves higher linear SVM classification accuracy than the benchmark. In addition, the proposed decoder structure is shown, in theory, to be a generic architecture that is able to reconstruct an arbitrary point cloud from a 2D grid. Our code is available at http://www.merl.com/research/license#FoldingNet

Accepted as a spotlight paper in CVPR'18

References in corpus (2)

• Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modeling
• Deep Convolutional Networks on Graph-Structured Data

Cited by in corpus (14)

• Point-BERT: Pre-training 3D Point Cloud Transformers with Masked Point Modeling
• Mining Point Cloud Local Structures by Kernel Correlation and Graph Pooling
• Patch-based Progressive 3D Point Set Upsampling
• PCN: Point Completion Network
• Coherent Point Drift Networks: Unsupervised Learning of Non-Rigid Point Set Registration
• Modeling Local Geometric Structure of 3D Point Clouds using Geo-CNN
• DAR-Net: Dynamic Aggregation Network for Semantic Scene Segmentation
• Learning Canonical Shape Space for Category-Level 6D Object Pose and Size Estimation
• Style-based Point Generator with Adversarial Rendering for Point Cloud Completion
• 3D Point Cloud Denoising via Deep Neural Network based Local Surface Estimation
• Real-time Soft Body 3D Proprioception via Deep Vision-based Sensing
• Rotation-Invariant Autoencoders for Signals on Spheres
• DeepTracking-Net: 3D Tracking with Unsupervised Learning of Continuous Flow
• Meta Deformation Network: Meta Functionals for Shape Correspondence