Interpretable Adversarial Perturbation in Input Embedding Space for Text

Abstract

Following great success in the image processing field, the idea of adversarial training has been applied to tasks in the natural language processing (NLP) field. One promising approach directly applies adversarial training developed in the image processing field to the input word embedding space instead of the discrete input space of texts. However, this approach abandons such interpretability as generating adversarial texts to significantly improve the performance of NLP tasks. This paper restores interpretability to such methods by restricting the directions of perturbations toward the existing words in the input embedding space. As a result, we can straightforwardly reconstruct each input with perturbations to an actual text by considering the perturbations to be the replacement of words in the sentence while maintaining or even improving the task performance.

8 pages, 4 figures

References in corpus (6)

• Distributed Representations of Sentences and Documents
• Distributional Smoothing with Virtual Adversarial Training
• Semi-supervised Convolutional Neural Networks for Text Categorization via Region Embedding
• Towards Crafting Text Adversarial Samples
• Deceiving Google's Perspective API Built for Detecting Toxic Comments
• Compositional Sequence Labeling Models for Error Detection in Learner Writing

Cited by in corpus (17)

• Adversarial Attacks on Deep Learning Models in Natural Language Processing: A Survey
• Adv-BERT: BERT is not robust on misspellings! Generating nature adversarial samples on BERT
• Adversarial Robustness of Deep Code Comment Generation
• Software Testing for Machine Learning
• Interpreting Deep Learning Models in Natural Language Processing: A Review
• Generating Natural Language Adversarial Examples on a Large Scale with Generative Models
• Interpretable Adversarial Training for Text
• To Transfer or Not to Transfer: Misclassification Attacks Against Transfer Learned Text Classifiers
• On The State of Data In Computer Vision: Human Annotations Remain Indispensable for Developing Deep Learning Models
• Developing Future Human-Centered Smart Cities: Critical Analysis of Smart City Security, Interpretability, and Ethical Challenges
• Interpretable and Trustworthy Deepfake Detection via Dynamic Prototypes
• Adversarial Attack and Defense of Structured Prediction Models
• Adversarial Black-Box Attacks On Text Classifiers Using Multi-Objective Genetic Optimization Guided By Deep Networks
• Mixture of Expert/Imitator Networks: Scalable Semi-supervised Learning Framework
• Improving Robustness of Malware Classifiers using Adversarial Strings Generated from Perturbed Latent Representations
• Evaluating Document Coherence Modelling
• A Diversity-Enhanced and Constraints-Relaxed Augmentation for Low-Resource Classification