Multi-Modal 3D Object Detection in Autonomous Driving: a Survey

Abstract

In this survey, we first introduce the background of popular sensors used for self-driving, their data properties, and the corresponding object detection algorithms. Next, we discuss existing datasets that can be used for evaluating multi-modal 3D object detection algorithms. Then we present a review of multi-modal fusion based 3D detection networks, taking a close look at their fusion stage, fusion input and fusion granularity, and how these design choices evolve with time and technology. After the review, we discuss open challenges as well as possible solutions. We hope that this survey can help researchers to get familiar with the field and embark on investigations in the area of multi-modal 3D object detection.

Accepted by International Journal of Computer Vision (IJCV)

References in corpus (12)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• How transferable are features in deep neural networks?
• Convolutional Neural Networks for Sentence Classification
• Deep Continuous Fusion for Multi-Sensor 3D Object Detection
• CenterFusion: Center-based Radar and Camera Fusion for 3D Object Detection
• IPOD: Intensive Point-based Object Detector for Point Cloud
• AFDet: Anchor Free One Stage 3D Object Detection
• TANet: Robust 3D Object Detection from Point Clouds with Triple Attention
• Multi-View Adaptive Fusion Network for 3D Object Detection
• 1st Place Solution for Waymo Open Dataset Challenge -- 3D Detection and Domain Adaptation
• Deep Learning on Radar Centric 3D Object Detection
• Pseudo-labeling for Scalable 3D Object Detection