Radar-Camera Fusion for Object Detection and Semantic Segmentation in Autonomous Driving: A Comprehensive Review
arXiv:2304.10410 · doi:10.1109/TIV.2023.3307157
Abstract
Driven by deep learning techniques, perception technology in autonomous driving has developed rapidly in recent years, enabling vehicles to accurately detect and interpret surrounding environment for safe and efficient navigation. To achieve accurate and robust perception capabilities, autonomous vehicles are often equipped with multiple sensors, making sensor fusion a crucial part of the perception system. Among these fused sensors, radars and cameras enable a complementary and cost-effective perception of the surrounding environment regardless of lighting and weather conditions. This review aims to provide a comprehensive guideline for radar-camera fusion, particularly concentrating on perception tasks related to object detection and semantic segmentation.Based on the principles of the radar and camera sensors, we delve into the data processing process and representations, followed by an in-depth analysis and summary of radar-camera fusion datasets. In the review of methodologies in radar-camera fusion, we address interrogative questions, including "why to fuse", "what to fuse", "where to fuse", "when to fuse", and "how to fuse", subsequently discussing various challenges and potential research directions within this domain. To ease the retrieval and comparison of datasets and fusion methods, we also provide an interactive website: https://radar-camera-fusion.github.io.
Accepted by IEEE Transactions on Intelligent Vehicles (T-IV)
References in corpus (25)
- Very Deep Convolutional Networks for Large-Scale Image Recognition
- YOLOv4: Optimal Speed and Accuracy of Object Detection
- Rethinking Atrous Convolution for Semantic Image Segmentation
- Methods for Interpreting and Understanding Deep Neural Networks
- YOLOv6: A Single-Stage Object Detection Framework for Industrial Applications
- CenterFusion: Center-based Radar and Camera Fusion for 3D Object Detection
- Rethinking Network Design and Local Geometry in Point Cloud: A Simple Residual MLP Framework
- TJ4DRadSet: A 4D Radar Dataset for Autonomous Driving
- Towards Domain-Independent and Real-Time Gesture Recognition Using mmWave Signal
- HRFormer: High-Resolution Transformer for Dense Prediction
- Automotive Radar Interference Mitigation Using Adaptive Noise Canceller
- LXL: LiDAR Excluded Lean 3D Object Detection with 4D Imaging Radar and Camera Fusion
- How much real data do we actually need: Analyzing object detection performance using synthetic and real data
- Resource-efficient Deep Neural Networks for Automotive Radar Interference Mitigation
- Radar Voxel Fusion for 3D Object Detection
- A Comparative Survey of Deep Active Learning
- Lawin Transformer: Improving Semantic Segmentation Transformer with Multi-Scale Representations via Large Window Attention
- Gaussian Radar Transformer for Semantic Segmentation in Noisy Radar Data
- RadarGNN: Transformation Invariant Graph Neural Network for Radar-based Perception
- Radar-Camera Sensor Fusion for Joint Object Detection and Distance Estimation in Autonomous Vehicles
- Improving Multi-Modal Learning with Uni-Modal Teachers
- Radar Image Reconstruction from Raw ADC Data using Parametric Variational Autoencoder with Domain Adaptation
- Improved Multi-Scale Grid Rendering of Point Clouds for Radar Object Detection Networks
- RadSegNet: A Reliable Approach to Radar Camera Fusion
- aiMotive Dataset: A Multimodal Dataset for Robust Autonomous Driving with Long-Range Perception
Cited by in corpus (9)
- LXL: LiDAR Excluded Lean 3D Object Detection with 4D Imaging Radar and Camera Fusion
- Exploring Radar Data Representations in Autonomous Driving: A Comprehensive Review
- Enhancing Cross-Dataset Performance of Distracted Driving Detection With Score Softmax Classifier And Dynamic Gaussian Smoothing Supervision
- RadarCNN: Learning-based Indoor Object Classification from IQ Imaging Radar Data
- A Low-Complexity PFA-Based Autofocus Algorithm for Automotive SAR
- On-the-Fly Interrogation of Mobile Passive Sensors from the Fusion of Optical and Radar Data
- A Resource Efficient Fusion Network for Object Detection in Bird's-Eye View using Camera and Raw Radar Data
- REFNet++: Multi-Task Efficient Fusion of Camera and Radar Sensor Data in Bird's-Eye Polar View
- Reflection-Aware Reasoning for Non-Line-of-Sight Pedestrian Localization