e-SNLI: Natural Language Inference with Natural Language Explanations

Abstract

In order for machine learning to garner widespread public adoption, models must be able to provide interpretable and robust explanations for their decisions, as well as learn from human-provided explanations at train time. In this work, we extend the Stanford Natural Language Inference dataset with an additional layer of human-annotated natural language explanations of the entailment relations. We further implement models that incorporate these explanations into their training process and output them at test time. We show how our corpus of explanations, which we call e-SNLI, can be used for various goals, such as obtaining full sentence justifications of a model's decisions, improving universal sentence representations and transferring to out-of-domain NLI datasets. Our dataset thus opens up a range of research directions for using natural language explanations, both for improving models and for asserting their trust.

NeurIPS 2018

Cited by in corpus (53)

• From Anecdotal Evidence to Quantitative Evaluation Methods: A Systematic Review on Evaluating Explainable AI
• Is ChatGPT better than Human Annotators? Potential and Limitations of ChatGPT in Explaining Implicit Hate Speech
• Post-hoc Interpretability for Neural NLP: A Survey
• Compositional Explanations of Neurons
• Local Interpretations for Explainable Natural Language Processing: A Survey
• Can I Trust the Explainer? Verifying Post-hoc Explanatory Methods
• Bridging the Transparency Gap: What Can Explainable AI Learn From the AI Act?
• Abductive Commonsense Reasoning
• Measuring and Improving Consistency in Pretrained Language Models
• A Survey on Explainability in Machine Reading Comprehension
• Explainable Machine Learning with Prior Knowledge: An Overview
• e-SNLI-VE: Corrected Visual-Textual Entailment with Natural Language Explanations
• Towards Explainable NLP: A Generative Explanation Framework for Text Classification
• Explaining Deep Neural Networks
• Explaining Black Box Predictions and Unveiling Data Artifacts through Influence Functions
• Thinking Aloud: Dynamic Context Generation Improves Zero-Shot Reasoning Performance of GPT-2
• Using Interactive Feedback to Improve the Accuracy and Explainability of Question Answering Systems Post-Deployment
• LIREx: Augmenting Language Inference with Relevant Explanation
• E-KAR: A Benchmark for Rationalizing Natural Language Analogical Reasoning
• Human Evaluation of Spoken vs. Visual Explanations for Open-Domain QA
• The Struggles of Feature-Based Explanations: Shapley Values vs. Minimal Sufficient Subsets
• Cross-Modal Conceptualization in Bottleneck Models
• Weakly Supervised Explainable Phrasal Reasoning with Neural Fuzzy Logic
• Learning Variational Word Masks to Improve the Interpretability of Neural Text Classifiers
• Are Training Resources Insufficient? Predict First Then Explain!
• OPT-R: Exploring the Role of Explanations in Finetuning and Prompting for Reasoning Skills of Large Language Models
• Leakage-Adjusted Simulatability: Can Models Generate Non-Trivial Explanations of Their Behavior in Natural Language?
• Evaluating and Characterizing Human Rationales
• Rissanen Data Analysis: Examining Dataset Characteristics via Description Length
• Learning from the Best: Rationalizing Prediction by Adversarial Information Calibration
• Do Natural Language Explanations Represent Valid Logical Arguments? Verifying Entailment in Explainable NLI Gold Standards
• ExpBERT: Representation Engineering with Natural Language Explanations
• FastIF: Scalable Influence Functions for Efficient Model Interpretation and Debugging
• You Can Do Better! If You Elaborate the Reason When Making Prediction
• On Sample Based Explanation Methods for NLP:Efficiency, Faithfulness, and Semantic Evaluation
• SelfExplain: A Self-Explaining Architecture for Neural Text Classifiers
• Explaining Neural Network Predictions on Sentence Pairs via Learning Word-Group Masks
• Explainability-aided Domain Generalization for Image Classification
• Order Constraints in Optimal Transport
• What Gets Echoed? Understanding the "Pointers" in Explanations of Persuasive Arguments
• LEAN-LIFE: A Label-Efficient Annotation Framework Towards Learning from Explanation
• Explaining the Road Not Taken
• Thermostat: A Large Collection of NLP Model Explanations and Analysis Tools
• multiPRover: Generating Multiple Proofs for Improved Interpretability in Rule Reasoning
• Machine Reading Comprehension using Case-based Reasoning
• R4C: A Benchmark for Evaluating RC Systems to Get the Right Answer for the Right Reason
• ExplaGraphs: An Explanation Graph Generation Task for Structured Commonsense Reasoning
• Does External Knowledge Help Explainable Natural Language Inference? Automatic Evaluation vs. Human Ratings
• Connecting Attributions and QA Model Behavior on Realistic Counterfactuals
• Towards Explainable Fact Checking
• Investigating the Effect of Natural Language Explanations on Out-of-Distribution Generalization in Few-shot NLI
• An Investigation of Language Model Interpretability via Sentence Editing
• To what extent do human explanations of model behavior align with actual model behavior?