3D-BEVIS: Bird's-Eye-View Instance Segmentation

Abstract

Recent deep learning models achieve impressive results on 3D scene analysis tasks by operating directly on unstructured point clouds. A lot of progress was made in the field of object classification and semantic segmentation. However, the task of instance segmentation is less explored. In this work, we present 3D-BEVIS, a deep learning framework for 3D semantic instance segmentation on point clouds. Following the idea of previous proposal-free instance segmentation approaches, our model learns a feature embedding and groups the obtained feature space into semantic instances. Current point-based methods scale linearly with the number of points by processing local sub-parts of a scene individually. However, to perform instance segmentation by clustering, globally consistent features are required. Therefore, we propose to combine local point geometry with global context information from an intermediate bird's-eye view representation.

arXiv commentcamera-ready version for GCPR '19

References in corpus (13)

• PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation
• Learning to Segment Object Candidates
• Associative Embedding: End-to-End Learning for Joint Detection and Grouping
• ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes
• Semantic Instance Segmentation with a Discriminative Loss Function
• Dynamic Graph CNN for Learning on Point Clouds
• Semantic Instance Segmentation via Deep Metric Learning
• Pixels to Graphs by Associative Embedding
• Simultaneous Detection and Segmentation
• Complex-YOLO: Real-time 3D Object Detection on Point Clouds
• Know What Your Neighbors Do: 3D Semantic Segmentation of Point Clouds
• 3D-SIS: 3D Semantic Instance Segmentation of RGB-D Scans
• Intel RealSense Stereoscopic Depth Cameras

Cited by in corpus (17)

• Deep Learning for Image and Point Cloud Fusion in Autonomous Driving: A Review
• Learning Object Bounding Boxes for 3D Instance Segmentation on Point Clouds
• STEm-Seg: Spatio-temporal Embeddings for Instance Segmentation in Videos
• PST: Plant segmentation transformer for 3D point clouds of rapeseed plants at the podding stage
• Deep Learning for 3D Point Clouds: A Survey
• Deep Learning Based 3D Segmentation: A Survey
• Learning Gaussian Instance Segmentation in Point Clouds
• PointGroup: Dual-Set Point Grouping for 3D Instance Segmentation
• 3D-SIS: 3D Semantic Instance Segmentation of RGB-D Scans
• Identifying Unknown Instances for Autonomous Driving
• DyCo3D: Robust Instance Segmentation of 3D Point Clouds through Dynamic Convolution
• RevealNet: Seeing Behind Objects in RGB-D Scans
• Joint prototype and coefficient prediction for 3D instance segmentation
• Learning and Memorizing Representative Prototypes for 3D Point Cloud Semantic and Instance Segmentation
• Dynamic Convolution for 3D Point Cloud Instance Segmentation
• Bi-Directional Attention for Joint Instance and Semantic Segmentation in Point Clouds
• Learning to Reconstruct and Segment 3D Objects