On the Explainability of Natural Language Processing Deep Models

Abstract

While there has been a recent explosion of work on ExplainableAI ExAI on deep models that operate on imagery and tabular data, textual datasets present new challenges to the ExAI community. Such challenges can be attributed to the lack of input structure in textual data, the use of word embeddings that add to the opacity of the models and the difficulty of the visualization of the inner workings of deep models when they are trained on textual data. Lately, methods have been developed to address the aforementioned challenges and present satisfactory explanations on Natural Language Processing (NLP) models. However, such methods are yet to be studied in a comprehensive framework where common challenges are properly stated and rigorous evaluation practices and metrics are proposed. Motivated to democratize ExAI methods in the NLP field, we present in this work a survey that studies model-agnostic as well as model-specific explainability methods on NLP models. Such methods can either develop inherently interpretable NLP models or operate on pre-trained models in a post-hoc manner. We make this distinction and we further decompose the methods into three categories according to what they explain: (1) word embeddings (input-level), (2) inner workings of NLP models (processing-level) and (3) models' decisions (output-level). We also detail the different evaluation approaches interpretability methods in the NLP field. Finally, we present a case-study on the well-known neural machine translation in an appendix and we propose promising future research directions for ExAI in the NLP field.

References in corpus (26)

• Efficient Estimation of Word Representations in Vector Space
• Neural Machine Translation by Jointly Learning to Align and Translate
• Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation
• Towards A Rigorous Science of Interpretable Machine Learning
• Methods for Interpreting and Understanding Deep Neural Networks
• Learning Important Features Through Propagating Activation Differences
• Visualizing and Understanding Recurrent Networks
• How to Explain Individual Classification Decisions
• Reasoning about Entailment with Neural Attention
• Opportunities and Challenges in Explainable Artificial Intelligence (XAI): A Survey
• Attention is not Explanation
• Understanding Neural Networks through Representation Erasure
• Go for a Walk and Arrive at the Answer: Reasoning Over Paths in Knowledge Bases using Reinforcement Learning
• Deep Unfolding: Model-Based Inspiration of Novel Deep Architectures
• A Survey of the State of Explainable AI for Natural Language Processing
• Comparative Experiments on Disambiguating Word Senses: An Illustration of the Role of Bias in Machine Learning
• What do you learn from context? Probing for sentence structure in contextualized word representations
• Cross-Sentence N-ary Relation Extraction with Graph LSTMs
• Explaining and Interpreting LSTMs
• Attention Interpretability Across NLP Tasks
• exBERT: A Visual Analysis Tool to Explore Learned Representations in Transformers Models
• Interpretable Recurrent Neural Networks Using Sequential Sparse Recovery
• Invariant Rationalization
• L-Shapley and C-Shapley: Efficient Model Interpretation for Structured Data
• Visualizing Attention in Transformer-Based Language Representation Models
• Commonsense Knowledge Graph Reasoning by Selection or Generation? Why?

Cited by in corpus (7)

• Explainable Artificial Intelligence (XAI) 2.0: A Manifesto of Open Challenges and Interdisciplinary Research Directions
• Generative AI in the Construction Industry: Opportunities & Challenges
• Large Language Models Streamline Automated Machine Learning for Clinical Studies
• Large Language Models and 3D Vision for Intelligent Robotic Perception and Autonomy
• A Comprehensive Survey on Self-Interpretable Neural Networks
• Leveraging Encoder-only Large Language Models for Mobile App Review Feature Extraction
• On the Evaluation of the Plausibility and Faithfulness of Sentiment Analysis Explanations