Natural Language Adversarial Defense through Synonym Encoding

Abstract

In the area of natural language processing, deep learning models are recently known to be vulnerable to various types of adversarial perturbations, but relatively few works are done on the defense side. Especially, there exists few effective defense method against the successful synonym substitution based attacks that preserve the syntactic structure and semantic information of the original text while fooling the deep learning models. We contribute in this direction and propose a novel adversarial defense method called Synonym Encoding Method (SEM). Specifically, SEM inserts an encoder before the input layer of the target model to map each cluster of synonyms to a unique encoding and trains the model to eliminate possible adversarial perturbations without modifying the network architecture or adding extra data. Extensive experiments demonstrate that SEM can effectively defend the current synonym substitution based attacks and block the transferability of adversarial examples. SEM is also easy and efficient to scale to large models and big datasets.

arXiv commentAccepted by UAI 2021, code is avaliable at https://github.com/JHL-HUST/SEM

References in corpus (16)

• Efficient Estimation of Word Representations in Vector Space
• Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examples
• Recurrent Neural Network for Text Classification with Multi-Task Learning
• Theoretically Principled Trade-off between Robustness and Accuracy
• Adversarially Robust Generalization Requires More Data
• Deep Text Classification Can be Fooled
• Adversarial Attacks on Deep Learning Models in Natural Language Processing: A Survey
• Are Labels Required for Improving Adversarial Robustness?
• Towards Crafting Text Adversarial Samples
• Low Frequency Adversarial Perturbation
• Improving the Generalization of Adversarial Training with Domain Adaptation
• Adversarially Robust Generalization Just Requires More Unlabeled Data
• Unlabeled Data Improves Adversarial Robustness
• Adversarial Texts with Gradient Methods
• Robustness Verification for Transformers
• Shielding Google's language toxicity model against adversarial attacks

Cited by in corpus (9)

• Contextualized Perturbation for Textual Adversarial Attack
• A Real-time Defense against Website Fingerprinting Attacks
• Adversarial Attacks and Defense on Texts: A Survey
• Learning to Attack: Towards Textual Adversarial Attacking in Real-world Situations
• Simple Contrastive Representation Adversarial Learning for NLP Tasks
• Defense of Word-level Adversarial Attacks via Random Substitution Encoding
• Towards Black-box Attacks on Deep Learning Apps
• Generating Watermarked Adversarial Texts
• Attacks against Ranking Algorithms with Text Embeddings: a Case Study on Recruitment Algorithms