Adv-BNN: Improved Adversarial Defense through Robust Bayesian Neural Network

Abstract

We present a new algorithm to train a robust neural network against adversarial attacks. Our algorithm is motivated by the following two ideas. First, although recent work has demonstrated that fusing randomness can improve the robustness of neural networks (Liu 2017), we noticed that adding noise blindly to all the layers is not the optimal way to incorporate randomness. Instead, we model randomness under the framework of Bayesian Neural Network (BNN) to formally learn the posterior distribution of models in a scalable way. Second, we formulate the mini-max problem in BNN to learn the best model distribution under adversarial attacks, leading to an adversarial-trained Bayesian neural net. Experiment results demonstrate that the proposed algorithm achieves state-of-the-art performance under strong attacks. On CIFAR-10 with VGG network, our model leads to 14\% accuracy improvement compared with adversarial training (Madry 2017) and random self-ensemble (Liu 2017) under PGD attack with $0.035$ distortion, and the gap becomes even larger on a subset of ImageNet.

arXiv commentCode will be made available at https://github.com/xuanqing94/BayesianDefense

References in corpus (10)

• Show, Attend and Tell: Neural Image Caption Generation with Visual Attention
• Intriguing properties of neural networks
• Towards Deep Learning Models Resistant to Adversarial Attacks
• Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examples
• Delving into Transferable Adversarial Examples and Black-box Attacks
• Adversarially Robust Generalization Requires More Data
• Robust Physical-World Attacks on Deep Learning Models
• Black-box Adversarial Attacks with Limited Queries and Information
• Learning with a Strong Adversary
• Towards Robust Neural Networks via Random Self-ensemble

Cited by in corpus (30)

• A Review of Uncertainty Quantification in Deep Learning: Techniques, Applications and Challenges
• Adversarial Machine Learning in Image Classification: A Survey Towards the Defender's Perspective
• Torchattacks: A PyTorch Repository for Adversarial Attacks
• A Review of Adversarial Attack and Defense for Classification Methods
• NATTACK: Learning the Distributions of Adversarial Examples for an Improved Black-Box Attack on Deep Neural Networks
• Being Bayesian, Even Just a Bit, Fixes Overconfidence in ReLU Networks
• Robustness of Bayesian Neural Networks to Gradient-Based Attacks
• The Limitations of Model Uncertainty in Adversarial Settings
• Adversarial Machine Learning: Bayesian Perspectives
• Benchmarking Adversarial Robustness
• Adversarial Neural Pruning with Latent Vulnerability Suppression
• A Simple Fine-tuning Is All You Need: Towards Robust Deep Learning Via Adversarial Fine-tuning
• Comment on "Adv-BNN: Improved Adversarial Defense through Robust Bayesian Neural Network"
• Robustifying Models Against Adversarial Attacks by Langevin Dynamics
• Sitatapatra: Blocking the Transfer of Adversarial Samples
• Adversarial Feature Augmentation and Normalization for Visual Recognition
• Bayesian Inference with Certifiable Adversarial Robustness
• Bayesian Inference Forgetting
• Adversarial Attack and Defense in Deep Ranking
• Gradient-Free Adversarial Attacks for Bayesian Neural Networks
• Weight-Covariance Alignment for Adversarially Robust Neural Networks
• Vulnerability Under Adversarial Machine Learning: Bias or Variance?
• A Geometrical Approach to Evaluate the Adversarial Robustness of Deep Neural Networks
• SPADE: A Spectral Method for Black-Box Adversarial Robustness Evaluation
• Resilience from Diversity: Population-based approach to harden models against adversarial attacks
• Probabilistic Safety for Bayesian Neural Networks
• Universal Physical Camouflage Attacks on Object Detectors
• Adversarial Ranking Attack and Defense
• The Effect of Prior Lipschitz Continuity on the Adversarial Robustness of Bayesian Neural Networks
• Defense Through Diverse Directions