Attention Meets Perturbations: Robust and Interpretable Attention with Adversarial Training

Abstract

Although attention mechanisms have been applied to a variety of deep learning models and have been shown to improve the prediction performance, it has been reported to be vulnerable to perturbations to the mechanism. To overcome the vulnerability to perturbations in the mechanism, we are inspired by adversarial training (AT), which is a powerful regularization technique for enhancing the robustness of the models. In this paper, we propose a general training technique for natural language processing tasks, including AT for attention (Attention AT) and more interpretable AT for attention (Attention iAT). The proposed techniques improved the prediction performance and the model interpretability by exploiting the mechanisms with AT. In particular, Attention iAT boosts those advantages by introducing adversarial perturbation, which enhances the difference in the attention of the sentences. Evaluation experiments with ten open datasets revealed that AT for attention mechanisms, especially Attention iAT, demonstrated (1) the best performance in nine out of ten tasks and (2) more interpretable attention (i.e., the resulting attention correlated more strongly with gradient-based word importance) for all tasks. Additionally, the proposed techniques are (3) much less dependent on perturbation size in AT. Our code is available at https://github.com/shunk031/attention-meets-perturbation

arXiv comment12 pages, 4 figures. Accepted by IEEE Access on Jun. 21, 2021

References in corpus (18)

• Adam: A Method for Stochastic Optimization
• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• Neural Machine Translation by Jointly Learning to Align and Translate
• Explaining and Harnessing Adversarial Examples
• Intriguing properties of neural networks
• Character-level Convolutional Networks for Text Classification
• Teaching Machines to Read and Comprehend
• A Structured Self-attentive Sentence Embedding
• Delving into Transferable Adversarial Examples and Black-box Attacks
• Attention is not Explanation
• Understanding Neural Networks through Representation Erasure
• Understanding Adversarial Training: Increasing Local Stability of Neural Nets through Robust Optimization
• Efficient Transformers: A Survey
• Character-Level Question Answering with Attention
• Interpreting Adversarially Trained Convolutional Neural Networks
• A Theoretical Framework for Robustness of (Deep) Classifiers against Adversarial Examples
• Interpretable Adversarial Training for Text
• What Do Adversarially Robust Models Look At?

Cited by in corpus (3)

• SATO: Stable Text-to-Motion Framework
• On Sensitivity of Deep Learning Based Text Classification Algorithms to Practical Input Perturbations
• Making Attention Mechanisms More Robust and Interpretable with Virtual Adversarial Training