RODNet: A Real-Time Radar Object Detection Network Cross-Supervised by Camera-Radar Fused Object 3D Localization
arXiv:2102.05150 · doi:10.1109/JSTSP.2021.3058895
Abstract
Various autonomous or assisted driving strategies have been facilitated through the accurate and reliable perception of the environment around a vehicle. Among the commonly used sensors, radar has usually been considered as a robust and cost-effective solution even in adverse driving scenarios, e.g., weak/strong lighting or bad weather. Instead of considering to fuse the unreliable information from all available sensors, perception from pure radar data becomes a valuable alternative that is worth exploring. In this paper, we propose a deep radar object detection network, named RODNet, which is cross-supervised by a camera-radar fused algorithm without laborious annotation efforts, to effectively detect objects from the radio frequency (RF) images in real-time. First, the raw signals captured by millimeter-wave radars are transformed to RF images in range-azimuth coordinates. Second, our proposed RODNet takes a sequence of RF images as the input to predict the likelihood of objects in the radar field of view (FoV). Two customized modules are also added to handle multi-chirp information and object relative motion. Instead of using human-labeled ground truth for training, the proposed RODNet is cross-supervised by a novel 3D localization of detected objects using a camera-radar fusion (CRF) strategy in the training stage. Finally, we propose a method to evaluate the object detection performance of the RODNet. Due to no existing public dataset available for our task, we create a new dataset, named CRUW, which contains synchronized RGB and RF image sequences in various driving scenarios. With intensive experiments, our proposed cross-supervised RODNet achieves 86% average precision and 88% average recall of object detection performance, which shows the robustness to noisy scenarios in various driving conditions.
arXiv commentIEEE Journal of Selected Topics in Signal Processing Special Issue on Recent Advances in Automotive Radar Signal Processing. arXiv admin note: text overlap with arXiv:2003.01816
References in corpus (14)
- Very Deep Convolutional Networks for Large-Scale Image Recognition
- Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
- YOLOv3: An Incremental Improvement
- Deep Multi-modal Object Detection and Semantic Segmentation for Autonomous Driving: Datasets, Methods, and Challenges
- Tracking without bells and whistles
- Self-supervised Visual Feature Learning with Deep Neural Networks: A Survey
- CNN based Road User Detection using the 3D Radar Cube
- Seeing Around Street Corners: Non-Line-of-Sight Detection and Tracking In-the-Wild Using Doppler Radar
- Experiments with mmWave Automotive Radar Test-bed
- MOANA: An Online Learned Adaptive Appearance Model for Robust Multiple Object Tracking in 3D
- Traffic-Aware Multi-Camera Tracking of Vehicles Based on ReID and Camera Link Model
- Vehicle classification based on convolutional networks applied to FM-CW radar signals
- Exploit the Connectivity: Multi-Object Tracking with TrackletNet
- IA-MOT: Instance-Aware Multi-Object Tracking with Motion Consistency
Cited by in corpus (12)
- Radar-Camera Fusion for Object Detection and Semantic Segmentation in Autonomous Driving: A Comprehensive Review
- Exploring Radar Data Representations in Autonomous Driving: A Comprehensive Review
- A recurrent CNN for online object detection on raw radar frames
- Improved Multi-Scale Grid Rendering of Point Clouds for Radar Object Detection Networks
- ProbRadarM3F: mmWave Radar based Human Skeletal Pose Estimation with Probability Map Guided Multi-Format Feature Fusion
- A Deep Automotive Radar Detector using the RaDelft Dataset
- BoostRad: Enhancing Object Detection by Boosting Radar Reflections
- Achieving Efficient and Realistic Full-Radar Simulations and Automatic Data Annotation by exploiting Ray Meta Data of a Radar Ray Tracing Simulator
- SparseRadNet: Sparse Perception Neural Network on Subsampled Radar Data
- A Resource Efficient Fusion Network for Object Detection in Bird's-Eye View using Camera and Raw Radar Data
- REFNet++: Multi-Task Efficient Fusion of Camera and Radar Sensor Data in Bird's-Eye Polar View
- Rethinking of Radar's Role: A Camera-Radar Dataset and Systematic Annotator via Coordinate Alignment