Learning Object Bounding Boxes for 3D Instance Segmentation on Point Clouds

Abstract

We propose a novel, conceptually simple and general framework for instance segmentation on 3D point clouds. Our method, called 3D-BoNet, follows the simple design philosophy of per-point multilayer perceptrons (MLPs). The framework directly regresses 3D bounding boxes for all instances in a point cloud, while simultaneously predicting a point-level mask for each instance. It consists of a backbone network followed by two parallel network branches for 1) bounding box regression and 2) point mask prediction. 3D-BoNet is single-stage, anchor-free and end-to-end trainable. Moreover, it is remarkably computationally efficient as, unlike existing approaches, it does not require any post-processing steps such as non-maximum suppression, feature sampling, clustering or voting. Extensive experiments show that our approach surpasses existing work on both ScanNet and S3DIS datasets while being approximately 10x more computationally efficient. Comprehensive ablation studies demonstrate the effectiveness of our design.

arXiv commentNeurIPS 2019 Spotlight. Code and data are available at https://github.com/Yang7879/3D-BoNet

References in corpus (7)

• Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
• Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation
• MASC: Multi-scale Affinity with Sparse Convolution for 3D Instance Segmentation
• 3D-BEVIS: Bird's-Eye-View Instance Segmentation
• SqueezeSeg: Convolutional Neural Nets with Recurrent CRF for Real-Time Road-Object Segmentation from 3D LiDAR Point Cloud
• Stochastic Optimization of Sorting Networks via Continuous Relaxations
• Fusing Bird View LIDAR Point Cloud and Front View Camera Image for Deep Object Detection

Cited by in corpus (36)

• RandLA-Net: Efficient Semantic Segmentation of Large-Scale Point Clouds
• SegGroup: Seg-Level Supervision for 3D Instance and Semantic Segmentation
• Advancements in Point Cloud-Based 3D Defect Detection and Classification for Industrial Systems: A Comprehensive Survey
• FPCC: Fast Point Cloud Clustering based Instance Segmentation for Industrial Bin-picking
• Deep Learning for 3D Point Clouds: A Survey
• CenterNet3D: An Anchor Free Object Detector for Point Cloud
• MaskGroup: Hierarchical Point Grouping and Masking for 3D Instance Segmentation
• Robotic Pick-and-Place With Uncertain Object Instance Segmentation and Shape Completion
• Learning Gaussian Instance Segmentation in Point Clouds
• PIG-Net: Inception based Deep Learning Architecture for 3D Point Cloud Segmentation
• Towards Semantic Segmentation of Urban-Scale 3D Point Clouds: A Dataset, Benchmarks and Challenges
• Object as Hotspots: An Anchor-Free 3D Object Detection Approach via Firing of Hotspots
• Exploring Data-Efficient 3D Scene Understanding with Contrastive Scene Contexts
• Unsupervised Point Cloud Pre-Training via Occlusion Completion
• Deep Learning-Based 3D Instance and Semantic Segmentation: A Review
• Panoptic 3D Scene Reconstruction From a Single RGB Image
• RevealNet: Seeing Behind Objects in RGB-D Scans
• BPNet: Bézier Primitive Segmentation on 3D Point Clouds
• 3DRM:Pair-wise relation module for 3D object detection
• 3D Object Detection and High-Resolution Traffic Parameters Extraction Using Low-Resolution LiDAR Data
• Joint prototype and coefficient prediction for 3D instance segmentation
• PatchFormer: An Efficient Point Transformer with Patch Attention
• Pri3D: Can 3D Priors Help 2D Representation Learning?
• Canonical Voting: Towards Robust Oriented Bounding Box Detection in 3D Scenes
• Scan2Part: Fine-grained and Hierarchical Part-level Understanding of Real-World 3D Scans
• Scalable 3D Panoptic Segmentation As Superpoint Graph Clustering
• PointBA: Towards Backdoor Attacks in 3D Point Cloud
• Point Cloud Instance Segmentation with Semi-supervised Bounding-Box Mining
• Dynamic Convolution for 3D Point Cloud Instance Segmentation
• Learning Scene Dynamics from Point Cloud Sequences
• SparseDet: Towards End-to-End 3D Object Detection
• Bi-Directional Attention for Joint Instance and Semantic Segmentation in Point Clouds
• Region-Transformer: Self-Attention Region Based Class-Agnostic Point Cloud Segmentation
• SASO: Joint 3D Semantic-Instance Segmentation via Multi-scale Semantic Association and Salient Point Clustering Optimization
• MaskBEV: Joint Object Detection and Footprint Completion for Bird's-eye View 3D Point Clouds
• Learning to Reconstruct and Segment 3D Objects