TextAttack: A Framework for Adversarial Attacks, Data Augmentation, and Adversarial Training in NLP

Abstract

While there has been substantial research using adversarial attacks to analyze NLP models, each attack is implemented in its own code repository. It remains challenging to develop NLP attacks and utilize them to improve model performance. This paper introduces TextAttack, a Python framework for adversarial attacks, data augmentation, and adversarial training in NLP. TextAttack builds attacks from four components: a goal function, a set of constraints, a transformation, and a search method. TextAttack's modular design enables researchers to easily construct attacks from combinations of novel and existing components. TextAttack provides implementations of 16 adversarial attacks from the literature and supports a variety of models and datasets, including BERT and other transformers, and all GLUE tasks. TextAttack also includes data augmentation and adversarial training modules for using components of adversarial attacks to improve model accuracy and robustness. TextAttack is democratizing NLP: anyone can try data augmentation and adversarial training on any model or dataset, with just a few lines of code. Code and tutorials are available at https://github.com/QData/TextAttack.

6 pages. More details are shared at https://github.com/QData/TextAttack

References in corpus (8)

• Universal Sentence Encoder
• Exploring the Limits of Language Modeling
• BAE: BERT-based Adversarial Examples for Text Classification
• Technical Report on the CleverHans v2.1.0 Adversarial Examples Library
• It's Morphin' Time! Combating Linguistic Discrimination with Inflectional Perturbations
• Seq2Sick: Evaluating the Robustness of Sequence-to-Sequence Models with Adversarial Examples
• BERT-ATTACK: Adversarial Attack Against BERT Using BERT
• Evading classifiers in discrete domains with provable optimality guarantees

Cited by in corpus (11)

• Data Augmentation Approaches in Natural Language Processing: A Survey
• TextFlint: Unified Multilingual Robustness Evaluation Toolkit for Natural Language Processing
• Back-Translated Task Adaptive Pretraining: Improving Accuracy and Robustness on Text Classification
• Grey-box Adversarial Attack And Defence For Sentiment Classification
• OpenMEVA: A Benchmark for Evaluating Open-ended Story Generation Metrics
• AF2-Mutation: Adversarial Sequence Mutations against AlphaFold2 on Protein Tertiary Structure Prediction
• A Simple and Strong Baseline for Universal Targeted Attacks on Siamese Visual Tracking
• Text Counterfactuals via Latent Optimization and Shapley-Guided Search
• "TL;DR:" Out-of-Context Adversarial Text Summarization and Hashtag Recommendation
• Perturbing Inputs for Fragile Interpretations in Deep Natural Language Processing
• Robustness Tests of NLP Machine Learning Models: Search and Semantically Replace