Faithful and Plausible Explanations of Medical Code Predictions

Abstract

Machine learning models that offer excellent predictive performance often lack the interpretability necessary to support integrated human machine decision-making. In clinical medicine and other high-risk settings, domain experts may be unwilling to trust model predictions without explanations. Work in explainable AI must balance competing objectives along two different axes: 1) Explanations must balance faithfulness to the model's decision-making with their plausibility to a domain expert. 2) Domain experts desire local explanations of individual predictions and global explanations of behavior in aggregate. We propose to train a proxy model that mimics the behavior of the trained model and provides fine-grained control over these trade-offs. We evaluate our approach on the task of assigning ICD codes to clinical notes to demonstrate that explanations from the proxy model are faithful and replicate the trained model behavior.

arXiv commentThe first two authors contributed equally

References in corpus (7)

• Scikit-learn: Machine Learning in Python
• Towards A Rigorous Science of Interpretable Machine Learning
• Explaining Explanations in AI
• The Hidden Assumptions Behind Counterfactual Explanations and Principal Reasons
• The Promise and Peril of Human Evaluation for Model Interpretability
• Fine-grained Sentiment Analysis with Faithful Attention
• Global Aggregations of Local Explanations for Black Box models