Learning Global Pairwise Interactions with Bayesian Neural Networks

Abstract

Estimating global pairwise interaction effects, i.e., the difference between the joint effect and the sum of marginal effects of two input features, with uncertainty properly quantified, is centrally important in science applications. We propose a non-parametric probabilistic method for detecting interaction effects of unknown form. First, the relationship between the features and the output is modelled using a Bayesian neural network, capable of representing complex interactions and principled uncertainty. Second, interaction effects and their uncertainty are estimated from the trained model. For the second step, we propose an intuitive global interaction measure: Bayesian Group Expected Hessian (GEH), which aggregates information of local interactions as captured by the Hessian. GEH provides a natural trade-off between type I and type II error and, moreover, comes with theoretical guarantees ensuring that the estimated interaction effects and their uncertainty can be improved by training a more accurate BNN. The method empirically outperforms available non-probabilistic alternatives on simulated and real-world data. Finally, we demonstrate its ability to detect interpretable interactions between higher-level features (at deeper layers of the neural network).

arXiv comment8 pages

References in corpus (9)

• Auto-Encoding Variational Bayes
• Axiomatic Attribution for Deep Networks
• Consistent Individualized Feature Attribution for Tree Ensembles
• Variational Dropout and the Local Reparameterization Trick
• Variational Dropout Sparsifies Deep Neural Networks
• Interpretation of Prediction Models Using the Input Gradient
• Dropout Feature Ranking for Deep Learning Models
• Global Aggregations of Local Explanations for Black Box models
• The Kernel Interaction Trick: Fast Bayesian Discovery of Pairwise Interactions in High Dimensions

Cited by in corpus (7)

• Explaining Deep Neural Networks and Beyond: A Review of Methods and Applications
• Explainable Artificial Intelligence for Bayesian Neural Networks: Towards trustworthy predictions of ocean dynamics
• Technical Note: Game-Theoretic Interactions of Different Orders
• Can We Faithfully Represent Masked States to Compute Shapley Values on a DNN?
• A Unified Game-Theoretic Interpretation of Adversarial Robustness
• Uncertainty-aware Sensitivity Analysis Using Rényi Divergences
• Relate and Predict: Structure-Aware Prediction with Jointly Optimized Neural DAG