Consensus-based Interpretable Deep Neural Networks with Application to Mortality Prediction

Abstract

Deep neural networks have achieved remarkable success in various challenging tasks. However, the black-box nature of such networks is not acceptable to critical applications, such as healthcare. In particular, the existence of adversarial examples and their overgeneralization to irrelevant, out-of-distribution inputs with high confidence makes it difficult, if not impossible, to explain decisions by such networks. In this paper, we analyze the underlying mechanism of generalization of deep neural networks and propose an ($n$, $k$) consensus algorithm which is insensitive to adversarial examples and can reliably reject out-of-distribution samples. Furthermore, the consensus algorithm is able to improve classification accuracy by using multiple trained deep neural networks. To handle the complexity of deep neural networks, we cluster linear approximations of individual models and identify highly correlated clusters among different models to capture feature importance robustly, resulting in improved interpretability. Motivated by the importance of building accurate and interpretable prediction models for healthcare, our experimental results on an ICU dataset show the effectiveness of our algorithm in enhancing both the prediction accuracy and the interpretability of deep neural network models on one-year patient mortality prediction. In particular, while the proposed method maintains similar interpretability as conventional shallow models such as logistic regression, it improves the prediction accuracy significantly.

arXiv comment8 pages, 6 figures

References in corpus (12)

• Towards A Rigorous Science of Interpretable Machine Learning
• Axiomatic Attribution for Deep Networks
• Deep EHR: A Survey of Recent Advances in Deep Learning Techniques for Electronic Health Record (EHR) Analysis
• Understanding Black-box Predictions via Influence Functions
• Towards Robust Interpretability with Self-Explaining Neural Networks
• Towards Understanding the Role of Over-Parametrization in Generalization of Neural Networks
• Learning and Generalization in Overparameterized Neural Networks, Going Beyond Two Layers
• Towards Understanding Generalization of Deep Learning: Perspective of Loss Landscapes
• Assessing the Local Interpretability of Machine Learning Models
• Evaluating Explanation Without Ground Truth in Interpretable Machine Learning
• The Relative Performance of Ensemble Methods with Deep Convolutional Neural Networks for Image Classification
• Exploring Weight Symmetry in Deep Neural Networks

Cited by in corpus (1)

• Towards Quantifying Intrinsic Generalization of Deep ReLU Networks