Bayesian Inference with Certifiable Adversarial Robustness
arXiv:2102.05289
Abstract
We consider adversarial training of deep neural networks through the lens of Bayesian learning, and present a principled framework for adversarial training of Bayesian Neural Networks (BNNs) with certifiable guarantees. We rely on techniques from constraint relaxation of non-convex optimisation problems and modify the standard cross-entropy error model to enforce posterior robustness to worst-case perturbations in -balls around input points. We illustrate how the resulting framework can be combined with methods commonly employed for approximate inference of BNNs. In an empirical investigation, we demonstrate that the presented approach enables training of certifiably robust models on MNIST, FashionMNIST and CIFAR-10 and can also be beneficial for uncertainty calibration. Our method is the first to directly train certifiable BNNs, thus facilitating their deployment in safety-critical applications.
Accepted AISTATS2021
References in corpus (15)
- Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms
- Intriguing properties of neural networks
- Wild Patterns: Ten Years After the Rise of Adversarial Machine Learning
- Weight Uncertainty in Neural Networks
- Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examples
- On the Effectiveness of Interval Bound Propagation for Training Verifiably Robust Models
- Uncertainty-Aware Reinforcement Learning for Collision Avoidance
- A Simple Baseline for Bayesian Uncertainty in Deep Learning
- Adv-BNN: Improved Adversarial Defense through Robust Bayesian Neural Network
- Fast and Scalable Bayesian Deep Learning by Weight-Perturbation in Adam
- The Description Length of Deep Learning Models
- Robustness of Bayesian Neural Networks to Gradient-Based Attacks
- The Limitations of Model Uncertainty in Adversarial Settings
- Bayesian Adversarial Spheres: Bayesian Inference and Adversarial Examples in a Noiseless Setting
- Handling the Positive-Definite Constraint in the Bayesian Learning Rule