CLOCs: Camera-LiDAR Object Candidates Fusion for 3D Object Detection
arXiv:2009.00784
Abstract
There have been significant advances in neural networks for both 3D object detection using LiDAR and 2D object detection using video. However, it has been surprisingly difficult to train networks to effectively use both modalities in a way that demonstrates gain over single-modality networks. In this paper, we propose a novel Camera-LiDAR Object Candidates (CLOCs) fusion network. CLOCs fusion provides a low-complexity multi-modal fusion framework that significantly improves the performance of single-modality detectors. CLOCs operates on the combined output candidates before Non-Maximum Suppression (NMS) of any 2D and any 3D detector, and is trained to leverage their geometric and semantic consistencies to produce more accurate final 3D and 2D detection results. Our experimental evaluation on the challenging KITTI object detection benchmark, including 3D and bird's eye view metrics, shows significant improvements, especially at long distance, over the state-of-the-art fusion based methods. At time of submission, CLOCs ranks the highest among all the fusion-based methods in the official KITTI leaderboard. We will release our code upon acceptance.
References in corpus (4)
- Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
- Deep Continuous Fusion for Multi-Sensor 3D Object Detection
- Frustum ConvNet: Sliding Frustums to Aggregate Local Point-Wise Features for Amodal 3D Object Detection
- Pseudo-LiDAR++: Accurate Depth for 3D Object Detection in Autonomous Driving
Cited by in corpus (7)
- Lidar Light Scattering Augmentation (LISA): Physics-based Simulation of Adverse Weather Conditions for 3D Object Detection
- SE-SSD: Self-Ensembling Single-Stage Object Detector From Point Cloud
- Pyramid R-CNN: Towards Better Performance and Adaptability for 3D Object Detection
- Uncertainty-Encoded Multi-Modal Fusion for Robust Object Detection in Autonomous Driving
- LiDAR R-CNN: An Efficient and Universal 3D Object Detector
- UrbanNet: Leveraging Urban Maps for Long Range 3D Object Detection
- Frustum Fusion: Pseudo-LiDAR and LiDAR Fusion for 3D Detection