Rationalizing Predictions by Adversarial Information Calibration

Abstract

Explaining the predictions of AI models is paramount in safety-critical applications, such as in legal or medical domains. One form of explanation for a prediction is an extractive rationale, i.e., a subset of features of an instance that lead the model to give its prediction on that instance. For example, the subphrase ``he stole the mobile phone'' can be an extractive rationale for the prediction of ``Theft''. Previous works on generating extractive rationales usually employ a two-phase model: a selector that selects the most important features (i.e., the rationale) followed by a predictor that makes the prediction based exclusively on the selected features. One disadvantage of these works is that the main signal for learning to select features comes from the comparison of the answers given by the predictor to the ground-truth answers. In this work, we propose to squeeze more information from the predictor via an information calibration method. More precisely, we train two models jointly: one is a typical neural model that solves the task at hand in an accurate but black-box manner, and the other is a selector-predictor model that additionally produces a rationale for its prediction. The first model is used as a guide for the second model. We use an adversarial technique to calibrate the information extracted by the two models such that the difference between them is an indicator of the missed or over-selected features. In addition, for natural language tasks, we propose a language-model-based regularizer to encourage the extraction of fluent rationales. Experimental results on a sentiment analysis task, a hate speech recognition task as well as on three tasks from the legal domain show the effectiveness of our approach to rationale extraction.

arXiv commentarXiv admin note: substantial text overlap with arXiv:2012.08884

References in corpus (27)

• Auto-Encoding Variational Bayes
• Distilling the Knowledge in a Neural Network
• Sequence to Sequence Learning with Neural Networks
• Categorical Reparameterization with Gumbel-Softmax
• Axiomatic Attribution for Deep Networks
• Striving for Simplicity: The All Convolutional Net
• The information bottleneck method
• Learning Important Features Through Propagating Activation Differences
• Understanding Black-box Predictions via Influence Functions
• Predictive learning via rule ensembles
• Progressive Neural Networks
• Interpretable classifiers using rules and Bayesian analysis: Building a better stroke prediction model
• How to Explain Individual Classification Decisions
• Neural GPUs Learn Algorithms
• What is Gab? A Bastion of Free Speech or an Alt-Right Echo Chamber?
• Fooling LIME and SHAP: Adversarial Attacks on Post hoc Explanation Methods
• Feature relevance quantification in explainable AI: A causal problem
• Interpreting Blackbox Models via Model Extraction
• Estimating Training Data Influence by Tracing Gradient Descent
• Preserving Causal Constraints in Counterfactual Explanations for Machine Learning Classifiers
• Algorithmic Recourse: from Counterfactual Explanations to Interventions
• A Game Theoretic Approach to Class-wise Selective Rationalization
• Hate begets Hate: A Temporal Study of Hate Speech
• Beyond Individualized Recourse: Interpretable and Interactive Summaries of Actionable Recourses
• Noise-adding Methods of Saliency Map as Series of Higher Order Partial Derivative
• Implementing Neural Turing Machines
• Are BLEU and Meaning Representation in Opposition?