Theoretical evidence for adversarial robustness through randomization

Abstract

This paper investigates the theory of robustness against adversarial attacks. It focuses on the family of randomization techniques that consist in injecting noise in the network at inference time. These techniques have proven effective in many contexts, but lack theoretical arguments. We close this gap by presenting a theoretical analysis of these approaches, hence explaining why they perform well in practice. More precisely, we make two new contributions. The first one relates the randomization rate to robustness to adversarial attacks. This result applies for the general family of exponential distributions, and thus extends and unifies the previous approaches. The second contribution consists in devising a new upper bound on the adversarial generalization gap of randomized neural networks. We support our theoretical claims with a set of experiments.

References in corpus (7)

• The Effectiveness of Data Augmentation in Image Classification using Deep Learning
• Synthesizing Robust Adversarial Examples
• Certified Adversarial Robustness via Randomized Smoothing
• On Evaluating Adversarial Robustness
• Parseval Networks: Improving Robustness to Adversarial Examples
• Adversarial examples from computational constraints
• On Pinsker's Type Inequalities and Csiszar's f-divergences. Part I: Second and Fourth-Order Inequalities

Cited by in corpus (12)

• A Closer Look at Accuracy vs. Robustness
• Robust Neural Networks using Randomized Adversarial Training
• Randomization matters. How to defend against strong adversarial attacks
• Calibrated Surrogate Losses for Adversarially Robust Classification
• Adversarial Robustness by Design through Analog Computing and Synthetic Gradients
• PointGuard: Provably Robust 3D Point Cloud Classification
• Towards Rapid and Robust Adversarial Training with One-Step Attacks
• Almost Tight L0-norm Certified Robustness of Top-k Predictions against Adversarial Perturbations
• Weight-Covariance Alignment for Adversarially Robust Neural Networks
• A Le Cam Type Bound for Adversarial Learning and Applications
• Noisy Feature Mixup
• Towards Optimal Randomized Strategies in Adversarial Example Game