Improving the Adversarial Robustness of NLP Models by Information Bottleneck

Abstract

Existing studies have demonstrated that adversarial examples can be directly attributed to the presence of non-robust features, which are highly predictive, but can be easily manipulated by adversaries to fool NLP models. In this study, we explore the feasibility of capturing task-specific robust features, while eliminating the non-robust ones by using the information bottleneck theory. Through extensive experiments, we show that the models trained with our information bottleneck-based method are able to achieve a significant improvement in robust accuracy, exceeding performances of all the previously reported defense methods while suffering almost no performance drop in clean accuracy on SST-2, AGNEWS and IMDB datasets.

References in corpus (7)

• Explaining and Harnessing Adversarial Examples
• Intriguing properties of neural networks
• Character-level Convolutional Networks for Text Classification
• The information bottleneck method
• InfoBERT: Improving Robustness of Language Models from An Information Theoretic Perspective
• Word Embedding Perturbation for Sentence Classification
• Defense against Adversarial Attacks in NLP via Dirichlet Neighborhood Ensemble

Cited by in corpus (2)

• Enhancing Multimodal Entity and Relation Extraction with Variational Information Bottleneck
• Improving the Robustness of Transformer-based Large Language Models with Dynamic Attention