Imitation Attacks and Defenses for Black-box Machine Translation Systems

Abstract

Adversaries may look to steal or attack black-box NLP systems, either for financial gain or to exploit model errors. One setting of particular interest is machine translation (MT), where models have high commercial value and errors can be costly. We investigate possible exploits of black-box MT systems and explore a preliminary defense against such threats. We first show that MT systems can be stolen by querying them with monolingual sentences and training models to imitate their outputs. Using simulated experiments, we demonstrate that MT model stealing is possible even when imitation models have different input data or architectures than their target models. Applying these ideas, we train imitation models that reach within 0.6 BLEU of three production MT systems on both high-resource and low-resource language pairs. We then leverage the similarity of our imitation models to transfer adversarial examples to the production systems. We use gradient-based attacks that expose inputs which lead to semantically-incorrect translations, dropped content, and vulgar model outputs. To mitigate these vulnerabilities, we propose a defense that modifies translation outputs in order to misdirect the optimization of imitation models. This defense degrades the adversary's BLEU score and attack success rate at some cost in the defender's BLEU and inference speed.

arXiv commentEMNLP 2020

References in corpus (14)

• Distilling the Knowledge in a Neural Network
• Transferability in Machine Learning: from Phenomena to Black-Box Attacks using Adversarial Samples
• Delving into Transferable Adversarial Examples and Black-box Attacks
• Black-box Adversarial Attacks with Limited Queries and Information
• On Adversarial Examples for Character-Level Neural Machine Translation
• Insertion Transformer: Flexible Sequence Generation via Insertion Operations
• Multilingual Neural Machine Translation with Knowledge Distillation
• Thieves on Sesame Street! Model Extraction of BERT-based APIs
• Diverse Beam Search: Decoding Diverse Solutions from Neural Sequence Models
• Prediction Poisoning: Towards Defenses Against DNN Model Stealing Attacks
• DAWN: Dynamic Adversarial Watermarking of Neural Networks
• A framework for the extraction of Deep Neural Networks by leveraging public data
• Defending Against Machine Learning Model Stealing Attacks Using Deceptive Perturbations
• Exploring the Robustness of NMT Systems to Nonsensical Inputs

Cited by in corpus (6)

• Model Extraction and Adversarial Transferability, Your BERT is Vulnerable!
• Model-Based Robust Deep Learning: Generalizing to Natural, Out-of-Distribution Data
• Towards Variable-Length Textual Adversarial Attacks
• Adversarial Semantic Collisions
• Putting words into the system's mouth: A targeted attack on neural machine translation using monolingual data poisoning
• Fully Synthetic Data Improves Neural Machine Translation with Knowledge Distillation