VIN: Voxel-based Implicit Network for Joint 3D Object Detection and Segmentation for Lidars

Abstract

A unified neural network structure is presented for joint 3D object detection and point cloud segmentation in this paper. We leverage rich supervision from both detection and segmentation labels rather than using just one of them. In addition, an extension based on single-stage object detectors is proposed based on the implicit function widely used in 3D scene and object understanding. The extension branch takes the final feature map from the object detection module as input, and produces an implicit function that generates semantic distribution for each point for its corresponding voxel center. We demonstrated the performance of our structure on nuScenes-lidarseg, a large-scale outdoor dataset. Our solution achieves competitive results against state-of-the-art methods in both 3D object detection and point cloud segmentation with little additional computation load compared with object detection solutions. The capability of efficient weakly supervision semantic segmentation of the proposed method is also validated by experiments.

arXiv commentTo be presented at BMVC 2021

References in corpus (13)

• Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
• PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space
• ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes
• Class-balanced Grouping and Sampling for Point Cloud 3D Object Detection
• Cylinder3D: An Effective 3D Framework for Driving-scene LiDAR Semantic Segmentation
• SalsaNext: Fast, Uncertainty-aware Semantic Segmentation of LiDAR Point Clouds for Autonomous Driving
• Center-based 3D Object Detection and Tracking
• Cylindrical and Asymmetrical 3D Convolution Networks for LiDAR Segmentation
• (AF)2-S3Net: Attentive Feature Fusion with Adaptive Feature Selection for Sparse Semantic Segmentation Network
• Exploring Data Augmentation for Multi-Modality 3D Object Detection
• Mcity Data Collection for Automated Vehicles Study
• One Thing One Click: A Self-Training Approach for Weakly Supervised 3D Semantic Segmentation
• LiDAR-based Panoptic Segmentation via Dynamic Shifting Network