InfoAT: Improving Adversarial Training Using the Information Bottleneck Principle

Abstract

Adversarial training (AT) has shown excellent high performance in defending against adversarial examples. Recent studies demonstrate that examples are not equally important to the final robustness of models during AT, that is, the so-called hard examples that can be attacked easily exhibit more influence than robust examples on the final robustness. Therefore, guaranteeing the robustness of hard examples is crucial for improving the final robustness of the model. However, defining effective heuristics to search for hard examples is still difficult. In this article, inspired by the information bottleneck (IB) principle, we uncover that an example with high mutual information of the input and its associated latent representation is more likely to be attacked. Based on this observation, we propose a novel and effective adversarial training method (InfoAT). InfoAT is encouraged to find examples with high mutual information and exploit them efficiently to improve the final robustness of models. Experimental results show that InfoAT achieves the best robustness among different datasets and models in comparison with several state-of-the-art methods.

arXiv commentPublished in: IEEE Transactions on Neural Networks and Learning Systems ( Early Access )

References in corpus (18)

• Explaining and Harnessing Adversarial Examples
• Intriguing properties of neural networks
• The information bottleneck method
• Rényi Divergence and Kullback-Leibler Divergence
• Theoretically Principled Trade-off between Robustness and Accuracy
• Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks
• Adversarial Risk and the Dangers of Evaluating Against Weak Attacks
• MagNet: a Two-Pronged Defense against Adversarial Examples
• Adversarial Weight Perturbation Helps Robust Generalization
• Learning Representations for Neural Network-Based Classification Using the Information Bottleneck Principle
• Are Labels Required for Improving Adversarial Robustness?
• Measuring multivariate redundant information with pointwise common change in surprisal
• Defense Against Adversarial Attacks Using Feature Scattering-based Adversarial Training
• Minimally distorted Adversarial Examples with a Fast Adaptive Boundary Attack
• Image Synthesis with a Single (Robust) Classifier
• Maximum-Entropy Adversarial Data Augmentation for Improved Generalization and Robustness
• Uncertainty in the Variational Information Bottleneck
• Towards Evaluating the Robustness of Deep Diagnostic Models by Adversarial Attack

Cited by in corpus (1)

• Survey on Adversarial Attack and Defense for Medical Image Analysis: Methods and Challenges