Optimizing for Interpretability in Deep Neural Networks with Tree Regularization

Abstract

Deep models have advanced prediction in many domains, but their lack of interpretability remains a key barrier to the adoption in many real world applications. There exists a large body of work aiming to help humans understand these black box functions to varying levels of granularity -- for example, through distillation, gradients, or adversarial examples. These methods however, all tackle interpretability as a separate process after training. In this work, we take a different approach and explicitly regularize deep models so that they are well-approximated by processes that humans can step-through in little time. Specifically, we train several families of deep neural networks to resemble compact, axis-aligned decision trees without significant compromises in accuracy. The resulting axis-aligned decision functions uniquely make tree regularized models easy for humans to interpret. Moreover, for situations in which a single, global tree is a poor estimator, we introduce a regional tree regularizer that encourages the deep model to resemble a compact, axis-aligned decision tree in predefined, human-interpretable contexts. Using intuitive toy examples as well as medical tasks for patients in critical care and with HIV, we demonstrate that this new family of tree regularizers yield models that are easier for humans to simulate than simpler L1 or L2 penalties without sacrificing predictive power.

arXiv commentarXiv admin note: substantial text overlap with arXiv:1908.04494, arXiv:1711.06178

References in corpus (11)

• Scikit-learn: Machine Learning in Python
• Sequence to Sequence Learning with Neural Networks
• Methods for Interpreting and Understanding Deep Neural Networks
• Understanding Black-box Predictions via Influence Functions
• Learning both Weights and Connections for Efficient Neural Networks
• From Softmax to Sparsemax: A Sparse Model of Attention and Multi-Label Classification
• Grad-CAM: Why did you say that?
• XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networks
• Harnessing Deep Neural Networks with Logic Rules
• Bayesian Dark Knowledge
• An unexpected unity among methods for interpreting model predictions