Learning for Vehicle-to-Vehicle Cooperative Perception under Lossy Communication

Abstract

Deep learning has been widely used in the perception (e.g., 3D object detection) of intelligent vehicle driving. Due to the beneficial Vehicle-to-Vehicle (V2V) communication, the deep learning based features from other agents can be shared to the ego vehicle so as to improve the perception of the ego vehicle. It is named as Cooperative Perception in the V2V research, whose algorithms have been dramatically advanced recently. However, all the existing cooperative perception algorithms assume the ideal V2V communication without considering the possible lossy shared features because of the Lossy Communication (LC) which is common in the complex real-world driving scenarios. In this paper, we first study the side effect (e.g., detection performance drop) by the lossy communication in the V2V Cooperative Perception, and then we propose a novel intermediate LC-aware feature fusion method to relieve the side effect of lossy communication by a LC-aware Repair Network (LCRN) and enhance the interaction between the ego vehicle and other vehicles by a specially designed V2V Attention Module (V2VAM) including intra-vehicle attention of ego vehicle and uncertainty-aware inter-vehicle attention. The extensive experiment on the public cooperative perception dataset OPV2V (based on digital-twin CARLA simulator) demonstrates that the proposed method is quite effective for the cooperative point cloud based 3D object detection under lossy V2V communication.

this paper was accepted by IEEE Transactions on Intelligent Vehicles

References in corpus (7)

• Decoupled Weight Decay Regularization
• Milestones in Autonomous Driving and Intelligent Vehicles: Survey of Surveys
• Cooperative Perception for 3D Object Detection in Driving Scenarios using Infrastructure Sensors
• COOPERNAUT: End-to-End Driving with Cooperative Perception for Networked Vehicles
• CoBEVT: Cooperative Bird's Eye View Semantic Segmentation with Sparse Transformers
• Where2comm: Communication-Efficient Collaborative Perception via Spatial Confidence Maps
• AutoCast: Scalable Infrastructure-less Cooperative Perception for Distributed Collaborative Driving

Cited by in corpus (6)

• Collaborative Perception in Autonomous Driving: Methods, Datasets and Challenges
• Vehicle-to-Everything Cooperative Perception for Autonomous Driving
• Egret: Reinforcement Mechanism for Sequential Computation Offloading in Edge Computing
• CooTest: An Automated Testing Approach for V2X Communication Systems
• Systematic Literature Review on Vehicular Collaborative Perception -- A Computer Vision Perspective
• Cooperative Safety Intelligence over V2X Networks: A Survey