Generating Natural Language Adversarial Examples on a Large Scale with Generative Models

Abstract

Today text classification models have been widely used. However, these classifiers are found to be easily fooled by adversarial examples. Fortunately, standard attacking methods generate adversarial texts in a pair-wise way, that is, an adversarial text can only be created from a real-world text by replacing a few words. In many applications, these texts are limited in numbers, therefore their corresponding adversarial examples are often not diverse enough and sometimes hard to read, thus can be easily detected by humans and cannot create chaos at a large scale. In this paper, we propose an end to end solution to efficiently generate adversarial texts from scratch using generative models, which are not restricted to perturbing the given texts. We call it unrestricted adversarial text generation. Specifically, we train a conditional variational autoencoder (VAE) with an additional adversarial loss to guide the generation of adversarial examples. Moreover, to improve the validity of adversarial texts, we utilize discrimators and the training framework of generative adversarial networks (GANs) to make adversarial texts consistent with real data. Experimental results on sentiment analysis demonstrate the scalability and efficiency of our method. It can attack text classification models with a higher success rate than existing methods, and provide acceptable quality for humans in the meantime.

References in corpus (13)

• Categorical Reparameterization with Gumbel-Softmax
• On the Properties of Neural Machine Translation: Encoder-Decoder Approaches
• Seeing stars: Exploiting class relationships for sentiment categorization with respect to rating scales
• Deep Text Classification Can be Fooled
• Generating Adversarial Malware Examples for Black-Box Attacks Based on GAN
• Adversarial Attacks on Deep Learning Models in Natural Language Processing: A Survey
• Towards Crafting Text Adversarial Samples
• Seq2Sick: Evaluating the Robustness of Sequence-to-Sequence Models with Adversarial Examples
• Adversarial Texts with Gradient Methods
• Techniques for Learning Binary Stochastic Feedforward Neural Networks
• Interpretable Adversarial Perturbation in Input Embedding Space for Text
• Fooling OCR Systems with Adversarial Text Images
• On Evaluation of Adversarial Perturbations for Sequence-to-Sequence Models

Cited by in corpus (2)

• Differentiable Language Model Adversarial Attacks on Categorical Sequence Classifiers
• Generating Semantically Valid Adversarial Questions for TableQA