Adversarial Semantic Collisions
arXiv:2011.04743
Abstract
We study semantic collisions: texts that are semantically unrelated but judged as similar by NLP models. We develop gradient-based approaches for generating semantic collisions and demonstrate that state-of-the-art models for many tasks which rely on analyzing the meaning and similarity of texts-- including paraphrase identification, document retrieval, response suggestion, and extractive summarization-- are vulnerable to semantic collisions. For example, given a target query, inserting a crafted collision into an irrelevant document can shift its retrieval rank from 1000 to top 3. We show how to generate semantic collisions that evade perplexity-based filtering and discuss other potential mitigations. Our code is available at https://github.com/csong27/collision-bert.
References in corpus (9)
- Teaching Machines to Read and Comprehend
- On Evaluating Adversarial Robustness
- Deep Text Classification Can be Fooled
- Personalizing Dialogue Agents: I have a dog, do you have pets too?
- Poly-encoders: Transformer Architectures and Pre-training Strategies for Fast and Accurate Multi-sentence Scoring
- Smart Reply: Automated Response Suggestion for Email
- Deceiving Google's Perspective API Built for Detecting Toxic Comments
- Imitation Attacks and Defenses for Black-box Machine Translation Systems
- To Transfer or Not to Transfer: Misclassification Attacks Against Transfer Learned Text Classifiers