Deep Text Classification Can be Fooled

Abstract

In this paper, we present an effective method to craft text adversarial samples, revealing one important yet underestimated fact that DNN-based text classifiers are also prone to adversarial sample attack. Specifically, confronted with different adversarial scenarios, the text items that are important for classification are identified by computing the cost gradients of the input (white-box attack) or generating a series of occluded test samples (black-box attack). Based on these items, we design three perturbation strategies, namely insertion, modification, and removal, to generate adversarial samples. The experiment results show that the adversarial samples generated by our method can successfully fool both state-of-the-art character-level and word-level DNN-based text classifiers. The adversarial samples can be perturbed to any desirable classes without compromising their utilities. At the same time, the introduced perturbation is difficult to be perceived.

arXiv comment8 pages

References in corpus (6)

• Explaining and Harnessing Adversarial Examples
• Intriguing properties of neural networks
• Character-level Convolutional Networks for Text Classification
• Adversarial Examples for Evaluating Reading Comprehension Systems
• Detecting Adversarial Image Examples in Deep Networks with Adaptive Noise Reduction
• Adversarial Texts with Gradient Methods

Cited by in corpus (93)

• BAE: BERT-based Adversarial Examples for Text Classification
• Synthetic and Natural Noise Both Break Neural Machine Translation
• Adversarial Attacks on Deep Learning Models in Natural Language Processing: A Survey
• Detecting Adversarial Image Examples in Deep Networks with Adaptive Noise Reduction
• Is BERT Really Robust? A Strong Baseline for Natural Language Attack on Text Classification and Entailment
• Reevaluating Adversarial Examples in Natural Language
• Adversarial Attacks Against Deep Generative Models on Data: A Survey
• Adv-BERT: BERT is not robust on misspellings! Generating nature adversarial samples on BERT
• Seq2Sick: Evaluating the Robustness of Sequence-to-Sequence Models with Adversarial Examples
• BERT-ATTACK: Adversarial Attack Against BERT Using BERT
• Towards Robustness Against Natural Language Word Substitutions
• Can Adversarial Weight Perturbations Inject Neural Backdoors?
• Black-box Generation of Adversarial Text Sequences to Evade Deep Learning Classifiers
• Attack Graph Convolutional Networks by Adding Fake Nodes
• Adversarial Robustness for Code
• Natural Language Adversarial Defense through Synonym Encoding
• Defense against Adversarial Attacks in NLP via Dirichlet Neighborhood Ensemble
• Greedy Attack and Gumbel Attack: Generating Adversarial Examples for Discrete Data
• Towards a Robust Deep Neural Network in Texts: A Survey
• Self-Explaining Structures Improve NLP Models
• Testing the limits of natural language models for predicting human language judgments
• AdvFlow: Inconspicuous Black-box Adversarial Attacks using Normalizing Flows
• Evading classifiers in discrete domains with provable optimality guarantees
• Contextualized Perturbation for Textual Adversarial Attack
• Say What I Want: Towards the Dark Side of Neural Dialogue Models
• Discrete Adversarial Attacks and Submodular Optimization with Applications to Text Classification
• Learning to Discriminate Perturbations for Blocking Adversarial Attacks in Text Classification
• Chat as Expected: Learning to Manipulate Black-box Neural Dialogue Models
• Generating Natural Language Adversarial Examples on a Large Scale with Generative Models
• Adversarial Attacks and Defense on Texts: A Survey
• Certified Robustness to Word Substitution Ranking Attack for Neural Ranking Models
• A Critical Reflection on the Use of Toxicity Detection Algorithms in Proactive Content Moderation Systems
• Universal Adversarial Perturbation for Text Classification
• Defending Against Backdoor Attacks in Natural Language Generation
• Defense against adversarial attacks on deep convolutional neural networks through nonlocal denoising
• AED: An black-box NLP classifier model attacker
• Learning to Attack: Towards Textual Adversarial Attacking in Real-world Situations
• To Transfer or Not to Transfer: Misclassification Attacks Against Transfer Learned Text Classifiers
• A Survey on Out-of-Distribution Evaluation of Neural NLP Models
• Re-evaluating ADEM: A Deeper Look at Scoring Dialogue Responses
• Generating Sentiment-Preserving Fake Online Reviews Using Neural Language Models and Their Human- and Machine-based Detection
• Adversarial Security Attacks and Perturbations on Machine Learning and Deep Learning Methods
• Adversary for Social Good: Leveraging Adversarial Attacks to Protect Personal Attribute Privacy
• Word Shape Matters: Robust Machine Translation with Visual Embedding
• ShufText: A Simple Black Box Approach to Evaluate the Fragility of Text Classification Models
• One word at a time: adversarial attacks on retrieval models
• Adversarial Attack on Network Embeddings via Supervised Network Poisoning
• Do We Need Online NLU Tools?
• Improving Robustness and Generality of NLP Models Using Disentangled Representations
• Negative Training for Neural Dialogue Response Generation
• Effective and Imperceptible Adversarial Textual Attack via Multi-objectivization
• Efficient Combinatorial Optimization for Word-level Adversarial Textual Attack
• Defense of Word-level Adversarial Attacks via Random Substitution Encoding
• T3: Tree-Autoencoder Constrained Adversarial Text Generation for Targeted Attack
• Differentiable Language Model Adversarial Attacks on Categorical Sequence Classifiers
• A Modified Word Saliency-Based Adversarial Attack on Text Classification Models
• IAE: Irony-based Adversarial Examples for Sentiment Analysis Systems
• Attacking Convolutional Neural Network using Differential Evolution
• Generating Semantically Valid Adversarial Questions for TableQA
• Universal Rules for Fooling Deep Neural Networks based Text Classification
• AF2-Mutation: Adversarial Sequence Mutations against AlphaFold2 on Protein Tertiary Structure Prediction
• Rewriting Meaningful Sentences via Conditional BERT Sampling and an application on fooling text classifiers
• Towards Variable-Length Textual Adversarial Attacks
• Playing to Learn Better: Repeated Games for Adversarial Learning with Multiple Classifiers
• Risk Management Framework for Machine Learning Security
• CollaborER: A Self-supervised Entity Resolution Framework Using Multi-features Collaboration
• Identifying Adversarial Sentences by Analyzing Text Complexity
• Geometry matters: Exploring language examples at the decision boundary
• Benchmarking Popular Classification Models' Robustness to Random and Targeted Corruptions
• R&R: Metric-guided Adversarial Sentence Generation
• Adversarial Semantic Collisions
• Challenge AI Mind: A Crowd System for Proactive AI Testing
• We Can Always Catch You: Detecting Adversarial Patched Objects WITH or WITHOUT Signature
• Evaluating Document Coherence Modelling
• Natural Adversarial Sentence Generation with Gradient-based Perturbation
• PDPGD: Primal-Dual Proximal Gradient Descent Adversarial Attack
• Robustness Tests of NLP Machine Learning Models: Search and Semantically Replace
• Attacks against Ranking Algorithms with Text Embeddings: a Case Study on Recruitment Algorithms
• Perturbing Inputs for Fragile Interpretations in Deep Natural Language Processing
• A Strong Baseline for Query Efficient Attacks in a Black Box Setting
• What's in a Name? Are BERT Named Entity Representations just as Good for any other Name?
• Multi-granularity Textual Adversarial Attack with Behavior Cloning
• Evaluating the Robustness of Neural Language Models to Input Perturbations
• On the Transferability of Adversarial Attacksagainst Neural Text Classifier
• Adversarial Attacks on Deep Models for Financial Transaction Records
• Frustratingly Poor Performance of Reading Comprehension Models on Non-adversarial Examples
• Self-Gradient Networks
• Handling Korean Out-of-Vocabulary Words with Phoneme Representation Learning
• AdvExpander: Generating Natural Language Adversarial Examples by Expanding Text
• Adversarial Machine Learning in Text Analysis and Generation
• Adversarially Robust and Explainable Model Compression with On-Device Personalization for Text Classification
• Adversarial Stylometry in the Wild: Transferable Lexical Substitution Attacks on Author Profiling
• SEPP: Similarity Estimation of Predicted Probabilities for Defending and Detecting Adversarial Text