Adversarial Examples for Evaluating Reading Comprehension Systems

Abstract

Standard accuracy metrics indicate that reading comprehension systems are making rapid progress, but the extent to which these systems truly understand language remains unclear. To reward systems with real language understanding abilities, we propose an adversarial evaluation scheme for the Stanford Question Answering Dataset (SQuAD). Our method tests whether systems can answer questions about paragraphs that contain adversarially inserted sentences, which are automatically generated to distract computer systems without changing the correct answer or misleading humans. In this adversarial setting, the accuracy of sixteen published models drops from an average of $75\%$ F1 score to $36\%$; when the adversary is allowed to add ungrammatical sequences of words, average accuracy on four models decreases further to $7\%$. We hope our insights will motivate the development of new models that understand language more precisely.

EMNLP 2017

References in corpus (12)

• SQuAD: 100,000+ Questions for Machine Comprehension of Text
• Machine Comprehension Using Match-LSTM and Answer Pointer
• ReasoNet: Learning to Stop Reading in Machine Comprehension
• Adversarial Learning for Neural Dialogue Generation
• Simple Black-Box Adversarial Perturbations for Deep Networks
• Learning Recurrent Span Representations for Extractive Question Answering
• Multi-Perspective Context Matching for Machine Comprehension
• Making Neural QA as Simple as Possible but not Simpler
• End-to-End Answer Chunk Extraction and Ranking for Reading Comprehension
• Exploring Question Understanding and Adaptation in Neural-Network-Based Question Answering
• Structural Embedding of Syntactic Trees for Machine Comprehension
• Ruminating Reader: Reasoning with Gated Multi-Hop Attention

Cited by in corpus (131)

• Adversarially Robust Generalization Requires More Data
• Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge
• Generating Natural Adversarial Examples
• Deep Text Classification Can be Fooled
• Deep Learning: A Critical Appraisal
• Adversarial Risk and the Dangers of Evaluating Against Weak Attacks
• TextBugger: Generating Adversarial Text Against Real-world Applications
• The Next Decade in AI: Four Steps Towards Robust Artificial Intelligence
• Pathologies of Neural Models Make Interpretations Difficult
• Adversarial Examples: Attacks and Defenses for Deep Learning
• Adversarial Attack on Graph Structured Data
• Towards Fast Computation of Certified Robustness for ReLU Networks
• InfoGraph: Unsupervised and Semi-supervised Graph-Level Representation Learning via Mutual Information Maximization
• Adversarial Attacks Against Medical Deep Learning Systems
• Measuring and Reducing Gendered Correlations in Pre-trained Models
• WT5?! Training Text-to-Text Models to Explain their Predictions
• Reevaluating Adversarial Examples in Natural Language
• Adversarial Training for Large Neural Language Models
• HotFlip: White-Box Adversarial Examples for Text Classification
• On the Value of Out-of-Distribution Testing: An Example of Goodhart's Law
• Seq2Sick: Evaluating the Robustness of Sequence-to-Sequence Models with Adversarial Examples
• BERT-ATTACK: Adversarial Attack Against BERT Using BERT
• Multilingual Extractive Reading Comprehension by Runtime Machine Translation
• DuReader: a Chinese Machine Reading Comprehension Dataset from Real-world Applications
• Adversarially Regularising Neural NLI Models to Integrate Logical Background Knowledge
• Reinforced Mnemonic Reader for Machine Reading Comprehension
• Unshuffling Data for Improved Generalization
• Stochastic Answer Networks for Machine Reading Comprehension
• Daedalus: Breaking Non-Maximum Suppression in Object Detection via Adversarial Examples
• Machine Reading Comprehension: a Literature Review
• Greedy Attack and Gumbel Attack: Generating Adversarial Examples for Discrete Data
• Software Testing for Machine Learning
• Efficient and Robust Question Answering from Minimal Context over Documents
• The Future of Misinformation Detection: New Perspectives and Trends
• Uncertainty-Aware Reliable Text Classification
• A Survey on Neural Machine Reading Comprehension
• Adversarial Defense Framework for Graph Neural Network
• Generating 3D Adversarial Point Clouds
• DANCin SEQ2SEQ: Fooling Text Classifiers with Adversarial Text Example Generation
• Better Robustness by More Coverage: Adversarial Training with Mixup Augmentation for Robust Fine-tuning
• Robust Machine Comprehension Models via Adversarial Training
• ConAML: Constrained Adversarial Machine Learning for Cyber-Physical Systems
• Discrete Adversarial Attacks and Submodular Optimization with Applications to Text Classification
• SGD-X: A Benchmark for Robust Generalization in Schema-Guided Dialogue Systems
• Learning by Abstraction: The Neural State Machine
• Adversarial TableQA: Attention Supervision for Question Answering on Tables
• Weaver: Deep Co-Encoding of Questions and Documents for Machine Reading
• Beyond Leaderboards: A survey of methods for revealing weaknesses in Natural Language Inference data and models
• Tunneling Neural Perception and Logic Reasoning through Abductive Learning
• What Makes Reading Comprehension Questions Easier?
• Android HIV: A Study of Repackaging Malware for Evading Machine-Learning Detection
• What Happened? Leveraging VerbNet to Predict the Effects of Actions in Procedural Text
• Metamorphic Relation Based Adversarial Attacks on Differentiable Neural Computer
• Attention-Guided Answer Distillation for Machine Reading Comprehension
• Improving Question Answering by Commonsense-Based Pre-Training
• Robust Multilingual Part-of-Speech Tagging via Adversarial Training
• Adversarial Attacks and Defense on Texts: A Survey
• Conditional BERT Contextual Augmentation
• Verification of Neural Network Control Policy Under Persistent Adversarial Perturbation
• Tracking State Changes in Procedural Text: A Challenge Dataset and Models for Process Paragraph Comprehension
• The Importance of Being Recurrent for Modeling Hierarchical Structure
• Adversarial Extreme Multi-label Classification
• Artificial Intelligence Strategies for National Security and Safety Standards
• Multi-Passage Machine Reading Comprehension with Cross-Passage Answer Verification
• Interactive Language Learning by Question Answering
• Universal Adversarial Perturbation for Text Classification
• Exploring the Robustness of NMT Systems to Nonsensical Inputs
• Not All Datasets Are Born Equal: On Heterogeneous Data and Adversarial Examples
• Fooling Vision and Language Models Despite Localization and Attention Mechanism
• Adversarial Attacks against Neural Networks in Audio Domain: Exploiting Principal Components
• Generating Adversarial Examples in Chinese Texts Using Sentence-Pieces
• Explicit Utilization of General Knowledge in Machine Reading Comprehension
• Breaking NLI Systems with Sentences that Require Simple Lexical Inferences
• Normative Conflicts and Shallow AI Alignment
• Evaluation of Generalizability of Neural Program Analyzers under Semantic-Preserving Transformations
• Identify Susceptible Locations in Medical Records via Adversarial Attacks on Deep Predictive Models
• QAInfomax: Learning Robust Question Answering System by Mutual Information Maximization
• An Empirical Evaluation of Adversarial Robustness under Transfer Learning
• Counterfactual Variable Control for Robust and Interpretable Question Answering
• Enhancing Answer Boundary Detection for Multilingual Machine Reading Comprehension
• Semantics Preserving Adversarial Learning
• Behavior Analysis of NLI Models: Uncovering the Influence of Three Factors on Robustness
• Strength in Numbers: Trading-off Robustness and Computation via Adversarially-Trained Ensembles
• Rethinking Generalization of Neural Models: A Named Entity Recognition Case Study
• ReviewQA: a relational aspect-based opinion reading dataset
• Exploring Neural Models for Parsing Natural Language into First-Order Logic
• Mapping to Declarative Knowledge for Word Problem Solving
• Evaluating Gender Bias in Natural Language Inference
• Visual Referring Expression Recognition: What Do Systems Actually Learn?
• Built-in Vulnerabilities to Imperceptible Adversarial Perturbations
• Poison Attacks against Text Datasets with Conditional Adversarially Regularized Autoencoder
• Nudge Attacks on Point-Cloud DNNs
• Improving Robustness and Generality of NLP Models Using Disentangled Representations
• Towards Improving Adversarial Training of NLP Models
• Design and Interpretation of Universal Adversarial Patches in Face Detection
• Can NLI Models Verify QA Systems' Predictions?
• Robust Reading Comprehension with Linguistic Constraints via Posterior Regularization
• CROP: Certifying Robust Policies for Reinforcement Learning through Functional Smoothing
• ReCO: A Large Scale Chinese Reading Comprehension Dataset on Opinion
• Analyzing Language Learned by an Active Question Answering Agent
• The relational processing limits of classic and contemporary neural network models of language processing
• Knowledge Based Machine Reading Comprehension
• Does it care what you asked? Understanding Importance of Verbs in Deep Learning QA System
• Does Robustness Improve Fairness? Approaching Fairness with Word Substitution Robustness Methods for Text Classification
• An Empirical Study of Content Understanding in Conversational Question Answering
• Generating Semantically Valid Adversarial Questions for TableQA
• Using Adversarial Attacks to Reveal the Statistical Bias in Machine Reading Comprehension Models
• Modeling Discriminative Representations for Out-of-Domain Detection with Supervised Contrastive Learning
• Commonsense knowledge adversarial dataset that challenges ELECTRA
• Adversarial Black-Box Attacks On Text Classifiers Using Multi-Objective Genetic Optimization Guided By Deep Networks
• Golden Grain: Building a Secure and Decentralized Model Marketplace for MLaaS
• Controlled Caption Generation for Images Through Adversarial Attacks
• Answer Generation through Unified Memories over Multiple Passages
• A Study of the Tasks and Models in Machine Reading Comprehension
• Geometry matters: Exploring language examples at the decision boundary
• Robustness Tests of NLP Machine Learning Models: Search and Semantically Replace
• SINVAD: Search-based Image Space Navigation for DNN Image Classifier Test Input Generation
• Adversarial Attacks for Multi-view Deep Models
• A3Net: Adversarial-and-Attention Network for Machine Reading Comprehension
• Imbalanced Sentiment Classification Enhanced with Discourse Marker
• Teaching Syntax by Adversarial Distraction
• Weighted Global Normalization for Multiple Choice Reading Comprehension over Long Documents
• Investigating Image Applications Based on Spatial-Frequency Transform and Deep Learning Techniques
• Adversarially robust change point detection
• Can Edge Probing Tasks Reveal Linguistic Knowledge in QA Models?
• Generating Watermarked Adversarial Texts
• Knowing More About Questions Can Help: Improving Calibration in Question Answering
• Sorting through the noise: Testing robustness of information processing in pre-trained language models
• Mitigating the Position Bias of Transformer Models in Passage Re-Ranking
• Adversarially Robust and Explainable Model Compression with On-Device Personalization for Text Classification
• PredCoin: Defense against Query-based Hard-label Attack