PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space

Abstract

Few prior works study deep learning on point sets. PointNet by Qi et al. is a pioneer in this direction. However, by design PointNet does not capture local structures induced by the metric space points live in, limiting its ability to recognize fine-grained patterns and generalizability to complex scenes. In this work, we introduce a hierarchical neural network that applies PointNet recursively on a nested partitioning of the input point set. By exploiting metric space distances, our network is able to learn local features with increasing contextual scales. With further observation that point sets are usually sampled with varying densities, which results in greatly decreased performance for networks trained on uniform densities, we propose novel set learning layers to adaptively combine features from multiple scales. Experiments show that our network called PointNet++ is able to learn deep point set features efficiently and robustly. In particular, results significantly better than state-of-the-art have been obtained on challenging benchmarks of 3D point clouds.

References in corpus (9)

• Adam: A Method for Stochastic Optimization
• Very Deep Convolutional Networks for Large-Scale Image Recognition
• PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation
• Spectral Networks and Locally Connected Networks on Graphs
• ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes
• Multi-view Convolutional Neural Networks for 3D Shape Recognition
• Order Matters: Sequence to sequence for sets
• SyncSpecCNN: Synchronized Spectral CNN for 3D Shape Segmentation
• OctNet: Learning Deep 3D Representations at High Resolutions

Cited by in corpus (123)

• Learning data driven discretizations for partial differential equations
• Tensor field networks: Rotation- and translation-equivariant neural networks for 3D point clouds
• PCPNET: Learning Local Shape Properties from Raw Point Clouds
• PointSIFT: A SIFT-like Network Module for 3D Point Cloud Semantic Segmentation
• PUFA-GAN: A Frequency-Aware Generative Adversarial Network for 3D Point Cloud Upsampling
• VoxelNet: End-to-End Learning for Point Cloud Based 3D Object Detection
• A review on deep learning techniques for 3D sensed data classification
• Directionally Constrained Fully Convolutional Neural Network For Airborne Lidar Point Cloud Classification
• Indoor Scene Understanding in 2.5/3D for Autonomous Agents: A Survey
• Point-GNN: Graph Neural Network for 3D Object Detection in a Point Cloud
• SO-Net: Self-Organizing Network for Point Cloud Analysis
• Neural Dual Contouring
• Cross-Modality Knowledge Distillation Network for Monocular 3D Object Detection
• 3D-SPS: Single-Stage 3D Visual Grounding via Referred Point Progressive Selection
• 3D Object Detection Using Scale Invariant and Feature Reweighting Networks
• Learning 3D Shape Completion under Weak Supervision
• Paris-CARLA-3D: A Real and Synthetic Outdoor Point Cloud Dataset for Challenging Tasks in 3D Mapping
• Domain Adaptation on Point Clouds via Geometry-Aware Implicits
• Segmentation of structural parts of rosebush plants with 3D point-based deep learning methods
• GIFS: Neural Implicit Function for General Shape Representation
• Domain adaptation based self-correction model for COVID-19 infection segmentation in CT images
• Density-Aware Convolutional Networks with Context Encoding for Airborne LiDAR Point Cloud Classification
• GRASP-Net: Geometric Residual Analysis and Synthesis for Point Cloud Compression
• PPFNet: Global Context Aware Local Features for Robust 3D Point Matching
• Large-Scale 3D Shape Reconstruction and Segmentation from ShapeNet Core55
• Advancing Additive Manufacturing through Deep Learning: A Comprehensive Review of Current Progress and Future Challenges
• 3D Point Cloud Classification and Segmentation using 3D Modified Fisher Vector Representation for Convolutional Neural Networks
• Point-Cloud Deep Learning of Porous Media for Permeability Prediction
• Probabilistic 3D Multi-Object Tracking for Autonomous Driving
• LiDAR-Based Place Recognition For Autonomous Driving: A Survey
• Imbalance Knowledge-Driven Multi-modal Network for Land-Cover Semantic Segmentation Using Images and LiDAR Point Clouds
• When Deep Learning Meets Data Alignment: A Review on Deep Registration Networks (DRNs)
• A Survey of Simultaneous Localization and Mapping with an Envision in 6G Wireless Networks
• Scan-based Semantic Segmentation of LiDAR Point Clouds: An Experimental Study
• TransformerFusion: Monocular RGB Scene Reconstruction using Transformers
• Deep Learning on 3D Semantic Segmentation: A Detailed Review
• Multi-view Incremental Segmentation of 3D Point Clouds for Mobile Robots
• Noise-resistant Deep Learning for Object Classification in 3D Point Clouds Using a Point Pair Descriptor
• VAT-Mart: Learning Visual Action Trajectory Proposals for Manipulating 3D ARTiculated Objects
• LiDAR Spoofing Meets the New-Gen: Capability Improvements, Broken Assumptions, and New Attack Strategies
• A Closer Look at Few-Shot 3D Point Cloud Classification
• CRA-PCN: Point Cloud Completion with Intra- and Inter-level Cross-Resolution Transformers
• Point Cloud Registration using Representative Overlapping Points
• Generating 3D Adversarial Point Clouds
• SoftPool++: An Encoder-Decoder Network for Point Cloud Completion
• Fully-Convolutional Point Networks for Large-Scale Point Clouds
• Centroid Transformers: Learning to Abstract with Attention
• See Eye to Eye: A Lidar-Agnostic 3D Detection Framework for Unsupervised Multi-Target Domain Adaptation
• 3D objects and scenes classification, recognition, segmentation, and reconstruction using 3D point cloud data: A review
• Learning Shape Representation on Sparse Point Clouds for Volumetric Image Segmentation
• GSIP: Green Semantic Segmentation of Large-Scale Indoor Point Clouds
• Learning 2D-3D Correspondences To Solve The Blind Perspective-n-Point Problem
• Semi-Supervised Semantic Mapping through Label Propagation with Semantic Texture Meshes
• Generative Models as Distributions of Functions
• Weighted Point Cloud Augmentation for Neural Network Training Data Class-Imbalance
• PaRot: Patch-Wise Rotation-Invariant Network via Feature Disentanglement and Pose Restoration
• LiDAR Odometry Methodologies for Autonomous Driving: A Survey
• Learning Second-Order Attentive Context for Efficient Correspondence Pruning
• Iterative Distance-Aware Similarity Matrix Convolution with Mutual-Supervised Point Elimination for Efficient Point Cloud Registration
• Pyramid R-CNN: Towards Better Performance and Adaptability for 3D Object Detection
• Parametric Surface Constrained Upsampler Network for Point Cloud
• Frame Averaging for Invariant and Equivariant Network Design
• No-Reference Point Cloud Quality Assessment via Weighted Patch Quality Prediction
• LMNet: Real-time Multiclass Object Detection on CPU using 3D LiDAR
• A Deep Learning Approach to Identify Rock Bolts in Complex 3D Point Clouds of Underground Mines Captured Using Mobile Laser Scanners
• PointSmile: Point Self-supervised Learning via Curriculum Mutual Information
• Inference, Learning and Attention Mechanisms that Exploit and Preserve Sparsity in Convolutional Networks
• MOVIN: Real-time Motion Capture using a Single LiDAR
• KTNet: Knowledge Transfer for Unpaired 3D Shape Completion
• Lidar Point Cloud Guided Monocular 3D Object Detection
• From CAD models to soft point cloud labels: An automatic annotation pipeline for cheaply supervised 3D semantic segmentation
• DuMLP-Pin: A Dual-MLP-dot-product Permutation-invariant Network for Set Feature Extraction
• Holistic Parameteric Reconstruction of Building Models from Point Clouds
• Point-Based Multi-View Stereo Network
• Multimodal Feature-Driven Deep Learning for the Prediction of Duck Body Dimensions and Weight
• DrivAer Transformer: A high-precision and fast prediction method for vehicle aerodynamic drag coefficient based on the DrivAerNet++ dataset
• TetraSphere: A Neural Descriptor for O(3)-Invariant Point Cloud Analysis
• DeepPoint: A Deep Learning Model for 3D Reconstruction in Point Clouds via mmWave Radar
• Object Segmentation of Cluttered Airborne LiDAR Point Clouds
• Effective Model Sparsification by Scheduled Grow-and-Prune Methods
• Self-supervised Learning of Rotation-invariant 3D Point Set Features using Transformer and its Self-distillation
• Few-Shot Meta-Learning on Point Cloud for Semantic Segmentation
• Lidar Panoptic Segmentation in an Open World
• An Enhanced Classification Method Based on Adaptive Multi-Scale Fusion for Long-tailed Multispectral Point Clouds
• PAPooling: Graph-based Position Adaptive Aggregation of Local Geometry in Point Clouds
• Automatic Segmentation of Aircraft Dents in Point Clouds
• VIN: Voxel-based Implicit Network for Joint 3D Object Detection and Segmentation for Lidars
• Point Cloud Audio Processing
• Vote from the Center: 6 DoF Pose Estimation in RGB-D Images by Radial Keypoint Voting
• Dense RGB-D semantic mapping with Pixel-Voxel neural network
• Multi-Grid Graph Neural Networks with Self-Attention for Computational Mechanics
• RMS-FlowNet++: Efficient and Robust Multi-Scale Scene Flow Estimation for Large-Scale Point Clouds
• EventNet: Asynchronous Recursive Event Processing
• Rotation-Invariant Completion Network
• Context-Aware Indoor Point Cloud Object Generation through User Instructions
• What can I do here? Leveraging Deep 3D saliency and geometry for fast and scalable multiple affordance detection
• Local Aggressive Adversarial Attacks on 3D Point Cloud
• Semantic Photometric Bundle Adjustment on Natural Sequences
• HairNet: Single-View Hair Reconstruction using Convolutional Neural Networks
• No GPU? No problem: an ultra fast 3D detection of road users with a simple proposal generator and energy-based out-of-distribution PointNets
• 3D Brain Reconstruction by Hierarchical Shape-Perception Network from a Single Incomplete Image
• No-reference geometry quality assessment for colorless point clouds via list-wise rank learning
• A Point Cloud Generative Model via Tree-Structured Graph Convolutions for 3D Brain Shape Reconstruction
• Weakly Supervised Point Clouds Transformer for 3D Object Detection
• DVI: Depth Guided Video Inpainting for Autonomous Driving
• Self-Parameterization Based Multi-Resolution Mesh Convolution Networks
• Occlusion Guided Scene Flow Estimation on 3D Point Clouds
• Structure Information is the Key: Self-Attention RoI Feature Extractor in 3D Object Detection
• Projective Manifold Gradient Layer for Deep Rotation Regression
• Scene Editing as Teleoperation: A Case Study in 6DoF Kit Assembly
• PointMixer: MLP-Mixer for Point Cloud Understanding
• VERAM: View-Enhanced Recurrent Attention Model for 3D Shape Classification
• Semantic Dense Reconstruction with Consistent Scene Segments
• Review on 6D Object Pose Estimation with the focus on Indoor Scene Understanding
• GSECnet: Ground Segmentation of Point Clouds for Edge Computing
• Irregularly Tabulated MLP for Fast Point Feature Embedding
• Generalized Zero-Shot Learning for Point Cloud Segmentation with Evidence-Based Dynamic Calibration
• Explainability-Aware One Point Attack for Point Cloud Neural Networks
• Using Adaptive Gradient for Texture Learning in Single-View 3D Reconstruction
• Panoptic Segmentation: A Review
• Surrogate Modelling for Injection Molding Processes using Machine Learning
• 3D Shapes Local Geometry Codes Learning with SDF
• Generating Band-Limited Adversarial Surfaces Using Neural Networks