Explaining and Improving Model Behavior with k Nearest Neighbor Representations

Abstract

Interpretability techniques in NLP have mainly focused on understanding individual predictions using attention visualization or gradient-based saliency maps over tokens. We propose using k nearest neighbor (kNN) representations to identify training examples responsible for a model's predictions and obtain a corpus-level understanding of the model's behavior. Apart from interpretability, we show that kNN representations are effective at uncovering learned spurious associations, identifying mislabeled examples, and improving the fine-tuned model's performance. We focus on Natural Language Inference (NLI) as a case study and experiment with multiple datasets. Our method deploys backoff to kNN for BERT and RoBERTa on examples with low model confidence without any update to the model parameters. Our results indicate that the kNN approach makes the finetuned model more robust to adversarial inputs.

References in corpus (12)

• BERTScore: Evaluating Text Generation with BERT
• Understanding Black-box Predictions via Influence Functions
• Deep k-Nearest Neighbors: Towards Confident, Interpretable and Robust Deep Learning
• Improving Neural Language Models with a Continuous Cache
• Right for the Wrong Reasons: Diagnosing Syntactic Heuristics in Natural Language Inference
• Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks
• A Game Theoretic Approach to Class-wise Selective Rationalization
• Fine-grained Sentiment Analysis with Faithful Attention
• Explaining Black Box Predictions and Unveiling Data Artifacts through Influence Functions
• Retrieve and Refine: Improved Sequence Generation Models For Dialogue
• EXS: Explainable Search Using Local Model Agnostic Interpretability
• Reinforced Curriculum Learning on Pre-trained Neural Machine Translation Models

Cited by in corpus (7)

• QA Dataset Explosion: A Taxonomy of NLP Resources for Question Answering and Reading Comprehension
• Interpreting Deep Learning Models in Natural Language Processing: A Review
• Nearest neighbour approaches for Emotion Detection in Tweets
• FastIF: Scalable Influence Functions for Efficient Model Interpretation and Debugging
• An Empirical Comparison of Instance Attribution Methods for NLP
• On Training Instance Selection for Few-Shot Neural Text Generation
• Cross-Domain Data Integration for Named Entity Disambiguation in Biomedical Text