Explainable Neural Networks based on Additive Index Models

Abstract

Machine Learning algorithms are increasingly being used in recent years due to their flexibility in model fitting and increased predictive performance. However, the complexity of the models makes them hard for the data analyst to interpret the results and explain them without additional tools. This has led to much research in developing various approaches to understand the model behavior. In this paper, we present the Explainable Neural Network (xNN), a structured neural network designed especially to learn interpretable features. Unlike fully connected neural networks, the features engineered by the xNN can be extracted from the network in a relatively straightforward manner and the results displayed. With appropriate regularization, the xNN provides a parsimonious explanation of the relationship between the features and the output. We illustrate this interpretable feature--engineering property on simulated examples.

10 pages, 8 figures

References in corpus (4)

• Distilling the Knowledge in a Neural Network
• Axiomatic Attribution for Deep Networks
• Locally Interpretable Models and Effects based on Supervised Partitioning (LIME-SUP)
• ActiVis: Visual Exploration of Industry-Scale Deep Neural Network Models

Cited by in corpus (24)

• The role of explainability in creating trustworthy artificial intelligence for health care: a comprehensive survey of the terminology, design choices, and evaluation strategies
• Explainable, Interpretable & Trustworthy AI for Intelligent Digital Twin: Case Study on Remaining Useful Life
• How can I choose an explainer? An Application-grounded Evaluation of Post-hoc Explanations
• On Interpretability of Artificial Neural Networks: A Survey
• A Comprehensive Survey of Machine Learning Applied to Radar Signal Processing
• On the Art and Science of Machine Learning Explanations
• Interpreting CNNs via Decision Trees
• Unwrapping The Black Box of Deep ReLU Networks: Interpretability, Diagnostics, and Simplification
• GAMI-Net: An Explainable Neural Network based on Generalized Additive Models with Structured Interactions
• Proposed Guidelines for the Responsible Use of Explainable Machine Learning
• Distilling a Deep Neural Network into a Takagi-Sugeno-Kang Fuzzy Inference System
• Bias, Fairness, and Accountability with AI and ML Algorithms
• Model Explainability in Deep Learning Based Natural Language Processing
• Enhancing Explainability of Neural Networks through Architecture Constraints
• A multi-component framework for the analysis and design of explainable artificial intelligence
• Model Interpretation: A Unified Derivative-based Framework for Nonparametric Regression and Supervised Machine Learning
• Explaining Neural Networks Semantically and Quantitatively
• On marginal feature attributions of tree-based models
• Interpretable CNNs for Object Classification
• Defining and Quantifying the Emergence of Sparse Concepts in DNNs
• Explaining AlphaGo: Interpreting Contextual Effects in Neural Networks
• Explainable Recommendation Systems by Generalized Additive Models with Manifest and Latent Interactions
• Explanatory Graphs for CNNs
• Verifiability and Predictability: Interpreting Utilities of Network Architectures for Point Cloud Processing