PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation
arXiv:1612.00593
Abstract
Point cloud is an important type of geometric data structure. Due to its irregular format, most researchers transform such data to regular 3D voxel grids or collections of images. This, however, renders data unnecessarily voluminous and causes issues. In this paper, we design a novel type of neural network that directly consumes point clouds and well respects the permutation invariance of points in the input. Our network, named PointNet, provides a unified architecture for applications ranging from object classification, part segmentation, to scene semantic parsing. Though simple, PointNet is highly efficient and effective. Empirically, it shows strong performance on par or even better than state of the art. Theoretically, we provide analysis towards understanding of what the network has learnt and why the network is robust with respect to input perturbation and corruption.
CVPR 2017
References in corpus (5)
Cited by in corpus (204)
- PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space
- A Survey of Deep Learning Techniques for Autonomous Driving
- A Review on Deep Learning Techniques Applied to Semantic Segmentation
- Tensor field networks: Rotation- and translation-equivariant neural networks for 3D point clouds
- Deep Continuous Fusion for Multi-Sensor 3D Object Detection
- 3DFeat-Net: Weakly Supervised Local 3D Features for Point Cloud Registration
- Deep learning for determining a near-optimal topological design without any iteration
- PointSIFT: A SIFT-like Network Module for 3D Point Cloud Semantic Segmentation
- Image Matching across Wide Baselines: From Paper to Practice
- Class-balanced Grouping and Sampling for Point Cloud 3D Object Detection
- Deep Learning for Generic Object Detection: A Survey
- DeepIM: Deep Iterative Matching for 6D Pose Estimation
- Low-Rank Pairwise Alignment Bilinear Network For Few-Shot Fine-Grained Image Classification
- Integrating Deep Learning into CAD/CAE System: Generative Design and Evaluation of 3D Conceptual Wheel
- A review on deep learning techniques for 3D sensed data classification
- AdvPC: Transferable Adversarial Perturbations on 3D Point Clouds
- Robust Attentional Aggregation of Deep Feature Sets for Multi-view 3D Reconstruction
- Joint 3D Proposal Generation and Object Detection from View Aggregation
- City3D: Large-Scale Building Reconstruction from Airborne LiDAR Point Clouds
- A Fully Convolutional Network for Semantic Labeling of 3D Point Clouds
- Directionally Constrained Fully Convolutional Neural Network For Airborne Lidar Point Cloud Classification
- Indoor Scene Understanding in 2.5/3D for Autonomous Agents: A Survey
- Two-Stage Mesh Deep Learning for Automated Tooth Segmentation and Landmark Localization on 3D Intraoral Scans
- SO-Net: Self-Organizing Network for Point Cloud Analysis
- Adaptive Graph Convolutional Neural Networks
- DenseFusion: 6D Object Pose Estimation by Iterative Dense Fusion
- Cross-Modality Knowledge Distillation Network for Monocular 3D Object Detection
- 3D Object Detection Using Scale Invariant and Feature Reweighting Networks
- SUM: A Benchmark Dataset of Semantic Urban Meshes
- PST: Plant segmentation transformer for 3D point clouds of rapeseed plants at the podding stage
- PU-Net: Point Cloud Upsampling Network
- Learning 3D Shape Completion under Weak Supervision
- From SLAM to Situational Awareness: Challenges and Survey
- Segmentation of structural parts of rosebush plants with 3D point-based deep learning methods
- 3D-BEVIS: Bird's-Eye-View Instance Segmentation
- Density-Aware Convolutional Networks with Context Encoding for Airborne LiDAR Point Cloud Classification
- Local Spectral Graph Convolution for Point Set Feature Learning
- Radar Voxel Fusion for 3D Object Detection
- S4G: Amodal Single-view Single-Shot SE(3) Grasp Detection in Cluttered Scenes
- PointFusion: Deep Sensor Fusion for 3D Bounding Box Estimation
- Forecast Network-Wide Traffic States for Multiple Steps Ahead: A Deep Learning Approach Considering Dynamic Non-Local Spatial Correlation and Non-Stationary Temporal Dependency
- Advancing Additive Manufacturing through Deep Learning: A Comprehensive Review of Current Progress and Future Challenges
- Deep SCNN-based Real-time Object Detection for Self-driving Vehicles Using LiDAR Temporal Data
- Classification of Point Cloud Scenes with Multiscale Voxel Deep Network
- Equivariant Energy Flow Networks for Jet Tagging
- Rotation-Invariant Local-to-Global Representation Learning for 3D Point Cloud
- ShaRF: Shape-conditioned Radiance Fields from a Single View
- FakeNews: GAN-based generation of realistic 3D volumetric data -- A systematic review and taxonomy
- Imbalance Knowledge-Driven Multi-modal Network for Land-Cover Semantic Segmentation Using Images and LiDAR Point Clouds
- Roadside Lidar Vehicle Detection and Tracking Using Range And Intensity Background Subtraction
- UCSG-Net -- Unsupervised Discovering of Constructive Solid Geometry Tree
- Voxel-FPN: multi-scale voxel feature aggregation in 3D object detection from point clouds
- SPP-Net: Deep Absolute Pose Regression with Synthetic Views
- DeepSpline: Data-Driven Reconstruction of Parametric Curves and Surfaces
- CPS++: Improving Class-level 6D Pose and Shape Estimation From Monocular Images With Self-Supervised Learning
- 'CADSketchNet' -- An Annotated Sketch dataset for 3D CAD Model Retrieval with Deep Neural Networks
- Autonomous Driving with Deep Learning: A Survey of State-of-Art Technologies
- PolyGNN: Polyhedron-based Graph Neural Network for 3D Building Reconstruction from Point Clouds
- H-CNN: Spatial Hashing Based CNN for 3D Shape Analysis
- PIG-Net: Inception based Deep Learning Architecture for 3D Point Cloud Segmentation
- A Survey of Simultaneous Localization and Mapping with an Envision in 6G Wireless Networks
- Fit4CAD: A point cloud benchmark for fitting simple geometric primitives in CAD objects
- Learning Hybrid Representations for Automatic 3D Vessel Centerline Extraction
- Deep neural networks for fast acquisition of aortic 3D pressure and velocity flow fields
- Decomposing 3D Scenes into Objects via Unsupervised Volume Segmentation
- Deep learning with 4D spatio-temporal data representations for OCT-based force estimation
- Learning to See Forces: Surgical Force Prediction with RGB-Point Cloud Temporal Convolutional Networks
- Creative Problem Solving in Artificially Intelligent Agents: A Survey and Framework
- RibSeg Dataset and Strong Point Cloud Baselines for Rib Segmentation from CT Scans
- A Closer Look at Few-Shot 3D Point Cloud Classification
- CRA-PCN: Point Cloud Completion with Intra- and Inter-level Cross-Resolution Transformers
- Multi-Features Guidance Network for partial-to-partial point cloud registration
- PC-U Net: Learning to Jointly Reconstruct and Segment the Cardiac Walls in 3D from CT Data
- Generating 3D Adversarial Point Clouds
- Rules of the Road: Predicting Driving Behavior with a Convolutional Model of Semantic Interactions
- SoftPool++: An Encoder-Decoder Network for Point Cloud Completion
- Xihe: A 3D Vision-based Lighting Estimation Framework for Mobile Augmented Reality
- AlignNet-3D: Fast Point Cloud Registration of Partially Observed Objects
- 2D and 3D Segmentation of uncertain local collagen fiber orientations in SHG microscopy
- Transformer-based Context Condensation for Boosting Feature Pyramids in Object Detection
- Flexible Supervised Autonomy for Exploration in Subterranean Environments
- PointSCNet: Point Cloud Structure and Correlation Learning Based on Space Filling Curve-Guided Sampling
- USIP: Unsupervised Stable Interest Point Detection from 3D Point Clouds
- A Generative Modeling Framework for Inferring Families of Biomechanical Constitutive Laws in Data-Sparse Regimes
- SHREC 2021: Retrieval and classification of protein surfaces equipped with physical and chemical properties
- Exchangeable deep neural networks for set-to-set matching and learning
- Anatomy-guided domain adaptation for 3D in-bed human pose estimation
- Semi-Supervised Semantic Mapping through Label Propagation with Semantic Texture Meshes
- Juggling With Representations: On the Information Transfer Between Imagery, Point Clouds, and Meshes for Multi-Modal Semantics
- Weighted Point Cloud Augmentation for Neural Network Training Data Class-Imbalance
- A Hierarchical Approach for Generating Descriptive Image Paragraphs
- SnapshotNet: Self-supervised Feature Learning for Point Cloud Data Segmentation Using Minimal Labeled Data
- PaRot: Patch-Wise Rotation-Invariant Network via Feature Disentanglement and Pose Restoration
- Correspondence-Free Point Cloud Registration with SO(3)-Equivariant Implicit Shape Representations
- FeaStNet: Feature-Steered Graph Convolutions for 3D Shape Analysis
- Inferring Point Clouds from Single Monocular Images by Depth Intermediation
- Pit30M: A Benchmark for Global Localization in the Age of Self-Driving Cars
- Active Perception using Light Curtains for Autonomous Driving
- Learning 3D-3D Correspondences for One-shot Partial-to-partial Registration
- Parametric Surface Constrained Upsampler Network for Point Cloud
- 3D Object Detection with Pointformer
- DeepPerimeter: Indoor Boundary Estimation from Posed Monocular Sequences
- Cross-modal feature fusion for robust point cloud registration with ambiguous geometry
- PointSmile: Point Self-supervised Learning via Curriculum Mutual Information
- ARCH++: Animation-Ready Clothed Human Reconstruction Revisited
- Disentangling Geometric Deformation Spaces in Generative Latent Shape Models
- Learning 3D Segment Descriptors for Place Recognition
- Points2Pix: 3D Point-Cloud to Image Translation using conditional Generative Adversarial Networks
- KTNet: Knowledge Transfer for Unpaired 3D Shape Completion
- Automated segmentation of microtomography imaging of Egyptian mummies
- Theoretical bounds on data requirements for the ray-based classification
- Bounce and Learn: Modeling Scene Dynamics with Real-World Bounces
- LiDAR Loop Closure Detection using Semantic Graphs with Graph Attention Networks
- Learning Dense Features for Point Cloud Registration Using a Graph Attention Network
- Weakly Supervised Learning of Multi-Object 3D Scene Decompositions Using Deep Shape Priors
- DuMLP-Pin: A Dual-MLP-dot-product Permutation-invariant Network for Set Feature Extraction
- PosDiffNet: Positional Neural Diffusion for Point Cloud Registration in a Large Field of View with Perturbations
- Learning 3D Shapes as Multi-Layered Height-maps using 2D Convolutional Networks
- Object Segmentation of Cluttered Airborne LiDAR Point Clouds
- Horizontally Fused Training Array: An Effective Hardware Utilization Squeezer for Training Novel Deep Learning Models
- Deep Learning on Radar Centric 3D Object Detection
- Set-to-Sequence Methods in Machine Learning: a Review
- Point-Based Multi-View Stereo Network
- DeepPoint: A Deep Learning Model for 3D Reconstruction in Point Clouds via mmWave Radar
- 3DRM:Pair-wise relation module for 3D object detection
- An Enhanced Classification Method Based on Adaptive Multi-Scale Fusion for Long-tailed Multispectral Point Clouds
- 3D Adversarial Augmentations for Robust Out-of-Domain Predictions
- TCFNet: Bidirectional face-bone transformation via a Transformer-based coarse-to-fine point movement network
- Conv-MPN: Convolutional Message Passing Neural Network for Structured Outdoor Architecture Reconstruction
- Lidar Panoptic Segmentation in an Open World
- Local Deep Implicit Functions for 3D Shape
- Planning Paths Through Unknown Space by Imagining What Lies Therein
- AMIL: Adversarial Multi Instance Learning for Human Pose Estimation
- Fast Geometric Surface based Segmentation of Point Cloud from Lidar Data
- Mesh Variational Autoencoders with Edge Contraction Pooling
- Automatic Segmentation of Aircraft Dents in Point Clouds
- Point Cloud Audio Processing
- Weakly Supervised Label Learning Flows
- Dense RGB-D semantic mapping with Pixel-Voxel neural network
- RMS-FlowNet++: Efficient and Robust Multi-Scale Scene Flow Estimation for Large-Scale Point Clouds
- A Large-Scale Evaluation of Shape-Aware Neighborhood Weights and Neighborhood Sizes
- PartNet: A Recursive Part Decomposition Network for Fine-grained and Hierarchical Shape Segmentation
- 3DContextNet: K-d Tree Guided Hierarchical Learning of Point Clouds Using Local and Global Contextual Cues
- Gap Completion in Point Cloud Scene occluded by Vehicles using SGC-Net
- Rotation-Invariant Completion Network
- PointAR: Efficient Lighting Estimation for Mobile Augmented Reality
- A Fast Point Cloud Ground Segmentation Approach Based on Coarse-To-Fine Markov Random Field
- What can I do here? Leveraging Deep 3D saliency and geometry for fast and scalable multiple affordance detection
- Learning to Play General Video-Games via an Object Embedding Network
- Developing Creative AI to Generate Sculptural Objects
- No-reference geometry quality assessment for colorless point clouds via list-wise rank learning
- Dual Grid Net: hand mesh vertex regression from single depth maps
- HairNet: Single-View Hair Reconstruction using Convolutional Neural Networks
- MPDCompress - Matrix Permutation Decomposition Algorithm for Deep Neural Network Compression
- TractoEmbed: Modular Multi-level Embedding framework for white matter tract segmentation
- No GPU? No problem: an ultra fast 3D detection of road users with a simple proposal generator and energy-based out-of-distribution PointNets
- Method for the generation of depth images for view-based shape retrieval of 3D CAD model from partial point cloud
- Deep-learning-based classification and retrieval of components of a process plant from segmented point clouds
- 3D Shape Segmentation with Geometric Deep Learning
- Automatic Labeled LiDAR Data Generation based on Precise Human Model
- Real-time Multiple People Hand Localization in 4D Point Clouds
- Learning a Hierarchical Latent-Variable Model of 3D Shapes
- Unsupervised learning with sparse space-and-time autoencoders
- Representation Learning of Point Cloud Upsampling in Global and Local Inputs
- Grasp Type Estimation for Myoelectric Prostheses using Point Cloud Feature Learning
- PointNLM: Point Nonlocal-Means for vegetation segmentation based on middle echo point clouds
- A New Neural Network Architecture Invariant to the Action of Symmetry Subgroups
- LGENet: Local and Global Encoder Network for Semantic Segmentation of Airborne Laser Scanning Point Clouds
- An End to End Network Architecture for Fundamental Matrix Estimation
- VERAM: View-Enhanced Recurrent Attention Model for 3D Shape Classification
- ImpliCity: City Modeling from Satellite Images with Deep Implicit Occupancy Fields
- Structured 2D Representation of 3D Data for Shape Processing
- Set Aggregation Network as a Trainable Pooling Layer
- Defo-Net: Learning Body Deformation using Generative Adversarial Networks
- Learning Single-Image Depth from Videos using Quality Assessment Networks
- Learning Quadrangulated Patches For 3D Shape Processing
- ATDN vSLAM: An all-through Deep Learning-Based Solution for Visual Simultaneous Localization and Mapping
- Multimodal trajectory forecasting based on discrete heat map
- U-mesh: Human Correspondence Matching with Mesh Convolutional Networks
- A Robust Loss for Point Cloud Registration
- A real-time global re-localization framework for 3D LiDAR SLAM
- Meta-Aggregator: Learning to Aggregate for 1-bit Graph Neural Networks
- Generalized Zero-Shot Learning for Point Cloud Segmentation with Evidence-Based Dynamic Calibration
- 3D-MOV: Audio-Visual LSTM Autoencoder for 3D Reconstruction of Multiple Objects from Video
- Guided Point Contrastive Learning for Semi-supervised Point Cloud Semantic Segmentation
- A Dynamic Keypoints Selection Network for 6DoF Pose Estimation
- A deep neural network for multi-species fish detection using multiple acoustic cameras
- Combining Image- and Geometric-based Deep Learning for Shape Regression: A Comparison to Pixel-level Methods for Segmentation in Chest X-Ray
- Point Cloud Colorization Based on Densely Annotated 3D Shape Dataset
- Registration Loss Learning for Deep Probabilistic Point Set Registration
- Set Augmented Triplet Loss for Video Person Re-Identification
- Learning Fine-Grained Segmentation of 3D Shapes without Part Labels
- Gram Regularization for Multi-view 3D Shape Retrieval
- DGGAN: Depth-image Guided Generative Adversarial Networks for Disentangling RGB and Depth Images in 3D Hand Pose Estimation
- Recurrent 3D Attentional Networks for End-to-End Active Object Recognition
- Neural Ray-Tracing: Learning Surfaces and Reflectance for Relighting and View Synthesis
- Meta Deformation Network: Meta Functionals for Shape Correspondence
- Prostate motion modelling using biomechanically-trained deep neural networks on unstructured nodes
- Keypoint Autoencoders: Learning Interest Points of Semantics
- Solving Large-Scale 0-1 Knapsack Problems and its Application to Point Cloud Resampling
- PointManifoldCut: Point-wise Augmentation in the Manifold for Point Clouds
- CAR -- Cityscapes Attributes Recognition A Multi-category Attributes Dataset for Autonomous Vehicles
- Symmetry Detection of Occluded Point Cloud Using Deep Learning
- Generating Band-Limited Adversarial Surfaces Using Neural Networks