Adversarial Black-Box Attacks On Text Classifiers Using Multi-Objective Genetic Optimization Guided By Deep Networks
arXiv:2011.03901
Abstract
We propose a novel genetic-algorithm technique that generates black-box adversarial examples which successfully fool neural network based text classifiers. We perform a genetic search with multi-objective optimization guided by deep learning based inferences and Seq2Seq mutation to generate semantically similar but imperceptible adversaries. We compare our approach with DeepWordBug (DWB) on SST and IMDB sentiment datasets by attacking three trained models viz. char-LSTM, word-LSTM and elmo-LSTM. On an average, we achieve an attack success rate of 65.67% for SST and 36.45% for IMDB across the three models showing an improvement of 49.48% and 101% respectively. Furthermore, our qualitative study indicates that 94% of the time, the users were not able to distinguish between an original and adversarial sample.
References in corpus (8)
- Explaining and Harnessing Adversarial Examples
- Pointer Sentinel Mixture Models
- Adversarial Examples for Evaluating Reading Comprehension Systems
- Adversarial Transformation Networks: Learning to Generate Adversarial Examples
- Relevance of Unsupervised Metrics in Task-Oriented Dialogue for Evaluating Natural Language Generation
- Houdini: Fooling Deep Structured Prediction Models
- UPSET and ANGRI : Breaking High Performance Image Classifiers
- Gender Bias in Contextualized Word Embeddings