3D Adversarial Augmentations for Robust Out-of-Domain Predictions

Abstract

Since real-world training datasets cannot properly sample the long tail of the underlying data distribution, corner cases and rare out-of-domain samples can severely hinder the performance of state-of-the-art models. This problem becomes even more severe for dense tasks, such as 3D semantic segmentation, where points of non-standard objects can be confidently associated to the wrong class. In this work, we focus on improving the generalization to out-of-domain data. We achieve this by augmenting the training set with adversarial examples. First, we learn a set of vectors that deform the objects in an adversarial fashion. To prevent the adversarial examples from being too far from the existing data distribution, we preserve their plausibility through a series of constraints, ensuring sensor-awareness and shapes smoothness. Then, we perform adversarial augmentation by applying the learned sample-independent vectors to the available objects when training a model. We conduct extensive experiments across a variety of scenarios on data from KITTI, Waymo, and CrashD for 3D object detection, and on data from SemanticKITTI, Waymo, and nuScenes for 3D semantic segmentation. Despite training on a standard single dataset, our approach substantially improves the robustness and generalization of both 3D object detection and 3D semantic segmentation methods to out-of-domain data.

37 pages, 12 figures

References in corpus (13)

• Explaining and Harnessing Adversarial Examples
• PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation
• Towards Deep Learning Models Resistant to Adversarial Attacks
• Adversarial Sensor Attack on LiDAR-based Perception in Autonomous Driving
• Generalizing to Unseen Domains via Adversarial Data Augmentation
• Moving Object Segmentation in 3D LiDAR Data: A Learning-based Approach Exploiting Sequential Data
• Invisible for both Camera and LiDAR: Security of Multi-Sensor Fusion based Perception in Autonomous Driving Under Physical-World Attacks
• AdvPC: Transferable Adversarial Perturbations on 3D Point Clouds
• Robust Monocular Depth Estimation under Challenging Conditions
• 3D-VField: Adversarial Augmentation of Point Clouds for Domain Generalization in 3D Object Detection
• ADef: an Iterative Algorithm to Construct Adversarial Deformations
• CertainNet: Sampling-free Uncertainty Estimation for Object Detection
• Segmenting Known Objects and Unseen Unknowns without Prior Knowledge

Cited by in corpus (1)

• Advancements in Point Cloud Data Augmentation for Deep Learning: A Survey