Improving Adversarial Robustness via Promoting Ensemble Diversity

Abstract

Though deep neural networks have achieved significant progress on various tasks, often enhanced by model ensemble, existing high-performance models can be vulnerable to adversarial attacks. Many efforts have been devoted to enhancing the robustness of individual networks and then constructing a straightforward ensemble, e.g., by directly averaging the outputs, which ignores the interaction among networks. This paper presents a new method that explores the interaction among individual networks to improve robustness for ensemble models. Technically, we define a new notion of ensemble diversity in the adversarial setting as the diversity among non-maximal predictions of individual members, and present an adaptive diversity promoting (ADP) regularizer to encourage the diversity, which leads to globally better robustness for the ensemble by making adversarial examples difficult to transfer among individual members. Our method is computationally efficient and compatible with the defense methods acting on individual networks. Empirical results on various datasets verify that our method can improve adversarial robustness while maintaining state-of-the-art accuracy on normal examples.

ICML 2019

Cited by in corpus (87)

• Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks
• Uncovering the Limits of Adversarial Training against Norm-Bounded Adversarial Examples
• On Adaptive Attacks to Adversarial Example Defenses
• Fixing Data Augmentation to Improve Adversarial Robustness
• Rethinking Softmax Cross-Entropy Loss for Adversarial Robustness
• Smooth Adversarial Training
• The Pitfalls of Simplicity Bias in Neural Networks
• Attacks Which Do Not Kill Training Make Adversarial Learning Stronger
• Boosting Adversarial Training with Hypersphere Embedding
• FDA3 : Federated Defense Against Adversarial Attacks for Cloud-Based IIoT Applications
• DVERGE: Diversifying Vulnerabilities for Enhanced Robust Generation of Ensembles
• BERT Loses Patience: Fast and Robust Inference with Early Exit
• Recent Advances in Adversarial Training for Adversarial Robustness
• Adversarially Robust Neural Architectures
• Adversarial Robustness: From Self-Supervised Pre-Training to Fine-Tuning
• Adversarial Distributional Training for Robust Deep Learning
• Tensor Normalization and Full Distribution Training
• Benchmarking Adversarial Robustness
• When NAS Meets Robustness: In Search of Robust Architectures against Adversarial Attacks
• What You See is Not What the Network Infers: Detecting Adversarial Examples Based on Semantic Contradiction
• Randomization matters. How to defend against strong adversarial attacks
• Exploring Memorization in Adversarial Training
• EMPIR: Ensembles of Mixed Precision Deep Networks for Increased Robustness against Adversarial Attacks
• Diverse Ensembles Improve Calibration
• Bag of Tricks for Adversarial Training
• ATHENA: A Framework based on Diverse Weak Defenses for Building Adversarial Defense
• Improving Robustness of Reinforcement Learning for Power System Control with Adversarial Training
• Category-wise Attack: Transferable Adversarial Examples for Anchor Free Object Detection
• Denoised Internal Models: a Brain-Inspired Autoencoder against Adversarial Attacks
• Optimal Transport as a Defense Against Adversarial Attacks
• Deep Neural Network Ensembles against Deception: Ensemble Diversity, Accuracy and Robustness
• TRS: Transferability Reduced Ensemble via Encouraging Gradient Diversity and Model Smoothness
• DiCOVA-Net: Diagnosing COVID-19 using Acoustics based on Deep Residual Network for the DiCOVA Challenge 2021
• On the Certified Robustness for Ensemble Models and Beyond
• Random Noise Defense Against Query-Based Black-Box Attacks
• Improving Certified Robustness via Statistical Learning with Logical Reasoning
• Guided Interpolation for Adversarial Training
• Multi-Agent Determinantal Q-Learning
• DICE: Diversity in Deep Ensembles via Conditional Redundancy Adversarial Estimation
• Robust Proxy: Improving Adversarial Robustness by Robust Proxy Learning
• Enhancing Certified Robustness via Smoothed Weighted Ensembling
• $n$-ML: Mitigating Adversarial Examples via Ensembles of Topologically Manipulated Classifiers
• Regional Adversarial Training for Better Robust Generalization
• Local Reweighting for Adversarial Training
• Indicators of Attack Failure: Debugging and Improving Optimization of Adversarial Examples
• Fooling LiDAR Perception via Adversarial Trajectory Perturbation
• Diversity and Generalization in Neural Network Ensembles
• Beware the Black-Box: on the Robustness of Recent Defenses to Adversarial Examples
• Automated Discovery of Adaptive Attacks on Adversarial Defenses
• Picket: Guarding Against Corrupted Data in Tabular Data during Learning and Inference
• Learning Diverse-Structured Networks for Adversarial Robustness
• Omni: Automated Ensemble with Unexpected Models against Adversarial Evasion Attack
• Ensemble Defense with Data Diversity: Weak Correlation Implies Strong Robustness
• Robustness via Cross-Domain Ensembles
• Ensemble-in-One: Learning Ensemble within Random Gated Networks for Enhanced Adversarial Robustness
• FedServing: A Federated Prediction Serving Framework Based on Incentive Mechanism
• On Procedural Adversarial Noise Attack And Defense
• Learning from Synthetic Data for Opinion-free Blind Image Quality Assessment in the Wild
• A Geometrical Approach to Evaluate the Adversarial Robustness of Deep Neural Networks
• Push Stricter to Decide Better: A Class-Conditional Feature Adaptive Framework for Improving Adversarial Robustness
• Mixture of Robust Experts (MoRE):A Robust Denoising Method towards multiple perturbations
• Understanding and Achieving Efficient Robustness with Adversarial Supervised Contrastive Learning
• Deep Neural Network Perception Models and Robust Autonomous Driving Systems
• Adversarial Attacks on ML Defense Models Competition
• Dynamic Defense Approach for Adversarial Robustness in Deep Neural Networks via Stochastic Ensemble Smoothed Model
• Robust Ensemble Model Training via Random Layer Sampling Against Adversarial Attack
• "What's in the box?!": Deflecting Adversarial Attacks by Randomly Deploying Adversarially-Disjoint Models
• The art of defense: letting networks fool the attacker
• Domain Invariant Adversarial Learning
• Model-Agnostic Meta-Attack: Towards Reliable Evaluation of Adversarial Robustness
• Evaluating Ensemble Robustness Against Adversarial Attacks
• MixDefense: A Defense-in-Depth Framework for Adversarial Example Detection Based on Statistical and Semantic Analysis
• CoG: a Two-View Co-training Framework for Defending Adversarial Attacks on Graph
• Get Fooled for the Right Reason: Improving Adversarial Robustness through a Teacher-guided Curriculum Learning Approach
• Rethinking Uncertainty in Deep Learning: Whether and How it Improves Robustness
• Training Provably Robust Models by Polyhedral Envelope Regularization
• Deep Learning-Based Autonomous Driving Systems: A Survey of Attacks and Defenses
• MadNet: Using a MAD Optimization for Defending Against Adversarial Attacks
• Where Does the Robustness Come from? A Study of the Transformation-based Ensemble Defence
• Introducing the DOME Activation Functions
• Adversarial defenses via a mixture of generators
• Measuring the Contribution of Multiple Model Representations in Detecting Adversarial Instances
• Towards Comprehensive Monocular Depth Estimation: Multiple Heads Are Better Than One
• Adversarial Training with Stochastic Weight Average
• Adaptive Perturbation for Adversarial Attack
• Accumulative Poisoning Attacks on Real-time Data
• Countering Adversarial Examples: Combining Input Transformation and Noisy Training