Adversarial Robustness of Deep Sensor Fusion Models

Abstract

We experimentally study the robustness of deep camera-LiDAR fusion architectures for 2D object detection in autonomous driving. First, we find that the fusion model is usually both more accurate, and more robust against single-source attacks than single-sensor deep neural networks. Furthermore, we show that without adversarial training, early fusion is more robust than late fusion, whereas the two perform similarly after adversarial training. However, we note that single-channel adversarial training of deep fusion is often detrimental even to robustness. Moreover, we observe cross-channel externalities, where single-channel adversarial training reduces robustness to attacks on the other channel. Additionally, we observe that the choice of adversarial model in adversarial training is critical: using attacks restricted to cars' bounding boxes is more effective in adversarial training and exhibits less significant cross-channel externalities. Finally, we find that joint-channel adversarial training helps mitigate many of the issues above, but does not significantly boost adversarial robustness.

References in corpus (7)

• YOLOv4: Optimal Speed and Accuracy of Object Detection
• NO Need to Worry about Adversarial Examples in Object Detection in Autonomous Vehicles
• Towards Robust LiDAR-based Perception in Autonomous Driving: General Black-box Adversarial Sensor Attack and Countermeasures
• Adversarial Objects Against LiDAR-Based Autonomous Driving Systems
• Exploring Adversarial Robustness of Multi-Sensor Perception Systems in Self Driving
• Investigating Vulnerability to Adversarial Examples on Multimodal Data Fusion in Deep Learning
• On Single Source Robustness in Deep Fusion Models