AllenNLP Interpret: A Framework for Explaining Predictions of NLP Models

Abstract

Neural NLP models are increasingly accurate but are imperfect and opaque---they break in counterintuitive ways and leave end users puzzled at their behavior. Model interpretation methods ameliorate this opacity by providing explanations for specific model predictions. Unfortunately, existing interpretation codebases make it difficult to apply these methods to new models and tasks, which hinders adoption for practitioners and burdens interpretability researchers. We introduce AllenNLP Interpret, a flexible framework for interpreting NLP models. The toolkit provides interpretation primitives (e.g., input gradients) for any AllenNLP model and task, a suite of built-in interpretation methods, and a library of front-end visualization components. We demonstrate the toolkit's flexibility and utility by implementing live demos for five interpretation methods (e.g., saliency maps and adversarial attacks) on a variety of models and tasks (e.g., masked language modeling using BERT and reading comprehension using BiDAF). These demos, alongside our code and tutorials, are available at https://allennlp.org/interpret .

EMNLP 2019 Demo

References in corpus (8)

• Towards A Rigorous Science of Interpretable Machine Learning
• Axiomatic Attribution for Deep Networks
• Bidirectional Attention Flow for Machine Comprehension
• Visualizing and Understanding Recurrent Networks
• SmoothGrad: removing noise by adding noise
• Technical Report on the CleverHans v2.1.0 Adversarial Examples Library
• Pathologies of Neural Models Make Interpretations Difficult
• Visualizing Attention in Transformer-Based Language Representation Models

Cited by in corpus (8)

• Reevaluating Adversarial Examples in Natural Language
• Interpreting Deep Learning Models in Natural Language Processing: A Review
• Pair the Dots: Jointly Examining Training History and Test Stimuli for Model Interpretability
• ShufText: A Simple Black Box Approach to Evaluate the Fragility of Text Classification Models
• Are Interpretations Fairly Evaluated? A Definition Driven Pipeline for Post-Hoc Interpretability
• WaLDORf: Wasteless Language-model Distillation On Reading-comprehension
• Logic Traps in Evaluating Attribution Scores
• TextAttack: Lessons learned in designing Python frameworks for NLP