Cooperative Perception for 3D Object Detection in Driving Scenarios using Infrastructure Sensors

Abstract

3D object detection is a common function within the perception system of an autonomous vehicle and outputs a list of 3D bounding boxes around objects of interest. Various 3D object detection methods have relied on fusion of different sensor modalities to overcome limitations of individual sensors. However, occlusion, limited field-of-view and low-point density of the sensor data cannot be reliably and cost-effectively addressed by multi-modal sensing from a single point of view. Alternatively, cooperative perception incorporates information from spatially diverse sensors distributed around the environment as a way to mitigate these limitations. This article proposes two schemes for cooperative 3D object detection using single modality sensors. The early fusion scheme combines point clouds from multiple spatially diverse sensing points of view before detection. In contrast, the late fusion scheme fuses the independently detected bounding boxes from multiple spatially diverse sensors. We evaluate the performance of both schemes, and their hybrid combination, using a synthetic cooperative dataset created in two complex driving scenarios, a T-junction and a roundabout. The evaluation shows that the early fusion approach outperforms late fusion by a significant margin at the cost of higher communication bandwidth. The results demonstrate that cooperative perception can recall more than 95% of the objects as opposed to 30% for single-point sensing in the most challenging scenario. To provide practical insights into the deployment of such system, we report how the number of sensors and their configuration impact the detection performance of the system.

arXiv comment13 pages, 4 tables, 7 figures. Published in IEEE Transactions on Intelligent Transportation Systems

References in corpus (3)

• Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
• Deep Multi-modal Object Detection and Semantic Segmentation for Autonomous Driving: Datasets, Methods, and Challenges
• F-Cooper: Feature based Cooperative Perception for Autonomous Vehicle Edge Computing System Using 3D Point Clouds

Cited by in corpus (16)

• Collaborative Perception in Autonomous Driving: Methods, Datasets and Challenges
• Intelligent Transportation Systems Using External Infrastructure: A Literature Survey
• Learning for Vehicle-to-Vehicle Cooperative Perception under Lossy Communication
• A Survey on Approximate Edge AI for Energy Efficient Autonomous Driving Services
• PillarGrid: Deep Learning-based Cooperative Perception for 3D Object Detection from Onboard-Roadside LiDAR
• Vehicle-to-Everything Cooperative Perception for Autonomous Driving
• Deep Learning-based Cooperative LiDAR Sensing for Improved Vehicle Positioning
• Cooperative Perception with Learning-Based V2V communications
• DUSA: Decoupled Unsupervised Sim2Real Adaptation for Vehicle-to-Everything Collaborative Perception
• Vehicular Cooperative Perception Through Action Branching and Federated Reinforcement Learning
• Collective PV-RCNN: A Novel Fusion Technique using Collective Detections for Enhanced Local LiDAR-Based Perception
• All You Need for Object Detection: From Pixels, Points, and Prompts to Next-Gen Fusion and Multimodal LLMs/VLMs in Autonomous Vehicles
• MVM3Det: A Novel Method for Multi-view Monocular 3D Detection
• Distributed Dynamic Map Fusion via Federated Learning for Intelligent Networked Vehicles
• HeCoFuse: Cross-Modal Complementary V2X Cooperative Perception with Heterogeneous Sensors
• Cooperative Safety Intelligence over V2X Networks: A Survey