Preserving Semantics in Textual Adversarial Attacks

Abstract

The growth of hateful online content, or hate speech, has been associated with a global increase in violent crimes against minorities [23]. Harmful online content can be produced easily, automatically and anonymously. Even though, some form of auto-detection is already achieved through text classifiers in NLP, they can be fooled by adversarial attacks. To strengthen existing systems and stay ahead of attackers, we need better adversarial attacks. In this paper, we show that up to 70% of adversarial examples generated by adversarial attacks should be discarded because they do not preserve semantics. We address this core weakness and propose a new, fully supervised sentence embedding technique called Semantics-Preserving-Encoder (SPE). Our method outperforms existing sentence encoders used in adversarial attacks by achieving 1.2x - 5.1x better real attack success rate. We release our code as a plugin that can be used in any existing adversarial attack to improve its quality and speed up its execution.

arXiv comment8 pages, 4 figures

References in corpus (21)

• Distributed Representations of Words and Phrases and their Compositionality
• Intriguing properties of neural networks
• Character-level Convolutional Networks for Text Classification
• BERTScore: Evaluating Text Generation with BERT
• Universal Sentence Encoder
• Seeing stars: Exploiting class relationships for sentiment categorization with respect to rating scales
• GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding
• BAE: BERT-based Adversarial Examples for Text Classification
• Generating Natural Adversarial Examples
• Adversarial Machine Learning at Scale
• TextBugger: Generating Adversarial Text Against Real-world Applications
• Adversarial Attacks on Deep Learning Models in Natural Language Processing: A Survey
• Efficient Natural Language Response Suggestion for Smart Reply
• Is BERT Really Robust? A Strong Baseline for Natural Language Attack on Text Classification and Entailment
• Reevaluating Adversarial Examples in Natural Language
• Towards Robustness Against Natural Language Word Substitutions
• Black-box Generation of Adversarial Text Sequences to Evade Deep Learning Classifiers
• Contextualized Perturbation for Textual Adversarial Attack
• Robust Encodings: A Framework for Combating Adversarial Typos
• Model Extraction and Adversarial Transferability, Your BERT is Vulnerable!
• T3: Tree-Autoencoder Constrained Adversarial Text Generation for Targeted Attack