Defending Against Backdoor Attacks in Natural Language Generation
arXiv:2106.01810
Abstract
The frustratingly fragile nature of neural network models make current natural language generation (NLG) systems prone to backdoor attacks and generate malicious sequences that could be sexist or offensive. Unfortunately, little effort has been invested to how backdoor attacks can affect current NLG models and how to defend against these attacks. In this work, by giving a formal definition of backdoor attack and defense, we investigate this problem on two important NLG tasks, machine translation and dialog generation. Tailored to the inherent nature of NLG models (e.g., producing a sequence of coherent words given contexts), we design defending strategies against attacks. We find that testing the backward probability of generating sources given targets yields effective defense performance against all different types of attacks, and is able to handle the {\it one-to-many} issue in many NLG tasks such as dialog generation. We hope that this work can raise the awareness of backdoor risks concealed in deep NLG systems and inspire more future work (both attack and defense) towards this direction.
arXiv commentTo appear at AAAI 2023
References in corpus (37)
- Sequence to Sequence Learning with Neural Networks
- Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation
- Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
- Convolutional Sequence to Sequence Learning
- BERTScore: Evaluating Text Generation with BERT
- XLNet: Generalized Autoregressive Pretraining for Language Understanding
- A Neural Conversational Model
- Understanding Black-box Predictions via Influence Functions
- Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoning
- BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chain
- Unified Language Model Pre-training for Natural Language Understanding and Generation
- Sequence Level Training with Recurrent Neural Networks
- Poisoning Attacks against Support Vector Machines
- DeBERTa: Decoding-enhanced BERT with Disentangled Attention
- Deep Reinforcement Learning for Dialogue Generation
- Deep Text Classification Can be Fooled
- FreeLB: Enhanced Adversarial Training for Natural Language Understanding
- Big Bird: Transformers for Longer Sequences
- Adversarial Learning for Neural Dialogue Generation
- Personalizing Dialogue Agents: I have a dog, do you have pets too?
- Input-Aware Dynamic Backdoor Attack
- TABOR: A Highly Accurate Approach to Inspecting and Restoring Trojan Backdoors in AI Systems
- Rethinking the Trigger of Backdoor Attack
- MetaPoison: Practical General-purpose Clean-label Data Poisoning
- HotFlip: White-Box Adversarial Examples for Text Classification
- Can Adversarial Weight Perturbations Inject Neural Backdoors?
- Just How Toxic is Data Poisoning? A Unified Benchmark for Backdoor and Data Poisoning Attacks
- Weight Poisoning Attacks on Pre-trained Models
- DeLighT: Deep and Light-weight Transformer
- Witches' Brew: Industrial Scale Data Poisoning via Gradient Matching
- Defense against Adversarial Attacks in NLP via Dirichlet Neighborhood Ensemble
- BAAAN: Backdoor Attacks Against Autoencoder and GAN-Based Machine Learning Models
- OpenViDial: A Large-Scale, Open-Domain Dialogue Dataset with Visual Contexts
- Be Careful about Poisoned Word Embeddings: Exploring the Vulnerability of the Embedding Layers in NLP Models
- Non-Autoregressive Neural Dialogue Generation
- Optimus: Organizing Sentences via Pre-trained Modeling of a Latent Space
- Teaching Machines to Converse