Reevaluating Adversarial Examples in Natural Language

Abstract

State-of-the-art attacks on NLP models lack a shared definition of a what constitutes a successful attack. We distill ideas from past work into a unified framework: a successful natural language adversarial example is a perturbation that fools the model and follows some linguistic constraints. We then analyze the outputs of two state-of-the-art synonym substitution attacks. We find that their perturbations often do not preserve semantics, and 38% introduce grammatical errors. Human surveys reveal that to successfully preserve semantics, we need to significantly increase the minimum cosine similarities between the embeddings of swapped words and between the sentence encodings of original and perturbed sentences.With constraints adjusted to better preserve semantics and grammaticality, the attack success rate drops by over 70 percentage points.

arXiv comment15 pages; 9 Tables; 5 Figures

References in corpus (11)

• Universal Sentence Encoder
• Generating Natural Adversarial Examples
• Deep Text Classification Can be Fooled
• Adversarial Examples for Evaluating Reading Comprehension Systems
• Towards Crafting Text Adversarial Samples
• Motivating the Rules of the Game for Adversarial Example Research
• Is BERT Really Robust? A Strong Baseline for Natural Language Attack on Text Classification and Entailment
• HotFlip: White-Box Adversarial Examples for Text Classification
• Seq2Sick: Evaluating the Robustness of Sequence-to-Sequence Models with Adversarial Examples
• Achieving Verified Robustness to Symbol Substitutions via Interval Bound Propagation
• AllenNLP Interpret: A Framework for Explaining Predictions of NLP Models

Cited by in corpus (10)

• Testing the limits of natural language models for predicting human language judgments
• Contextualized Perturbation for Textual Adversarial Attack
• Searching for a Search Method: Benchmarking Search Algorithms for Generating NLP Adversarial Examples
• Generating Adversarial Examples in Chinese Texts Using Sentence-Pieces
• Towards Improving Adversarial Training of NLP Models
• Preserving Semantics in Textual Adversarial Attacks
• Grey-box Adversarial Attack And Defence For Sentiment Classification
• Gradient-based Adversarial Attacks against Text Transformers
• Towards Variable-Length Textual Adversarial Attacks
• Searching for an Effective Defender: Benchmarking Defense against Adversarial Word Substitution