A PAC-Bayes Analysis of Adversarial Robustness

Abstract

We propose the first general PAC-Bayesian generalization bounds for adversarial robustness, that estimate, at test time, how much a model will be invariant to imperceptible perturbations in the input. Instead of deriving a worst-case analysis of the risk of a hypothesis over all the possible perturbations, we leverage the PAC-Bayesian framework to bound the averaged risk on the perturbations for majority votes (over the whole class of hypotheses). Our theoretically founded analysis has the advantage to provide general bounds (i) that are valid for any kind of attacks (i.e., the adversarial attacks), (ii) that are tight thanks to the PAC-Bayesian framework, (iii) that can be directly minimized during the learning phase to obtain a robust model on different attacks at test time.

References in corpus (12)

• Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms
• Certified Adversarial Robustness via Randomized Smoothing
• Provably Robust Deep Learning via Adversarially Trained Smoothed Classifiers
• Lipschitz-Margin Training: Scalable Certification of Perturbation Invariance for Deep Neural Networks
• Rademacher Complexity for Adversarially Robust Generalization
• Uniform convergence may be unable to explain generalization in deep learning
• Risk Bounds for the Majority Vote: From a PAC-Bayesian Analysis to a Learning Algorithm
• Generalizable Adversarial Training via Spectral Normalization
• VC Classes are Adversarially Robustly Learnable, but Only Improperly
• Chromatic PAC-Bayes Bounds for Non-IID Data: Applications to Ranking and Stationary $\beta$-Mixing Processes
• Learning Gaussian Processes by Minimizing PAC-Bayesian Generalization Bounds
• On the role of data in PAC-Bayes bounds