Interpretable Neural Networks based classifiers for categorical inputs

Abstract

Because of the pervasive usage of Neural Networks in human sensitive applications, their interpretability is becoming an increasingly important topic in machine learning. In this work we introduce a simple way to interpret the output function of a neural network classifier that take as input categorical variables. By exploiting a mapping between a neural network classifier and a physical energy model, we show that in these cases each layer of the network, and the logits layer in particular, can be expanded as a sum of terms that account for the contribution to the classification of each input pattern. For instance, at the first order, the expansion considers just the linear relation between input features and output while at the second order pairwise dependencies between input features are also accounted for. The analysis of the contributions of each pattern, after an appropriate gauge transformation, is presented in two cases where the effectiveness of the method can be appreciated.

arXiv comment11 pages, 4 figures

References in corpus (6)

• Towards A Rigorous Science of Interpretable Machine Learning
• Direct-coupling analysis of residue co-evolution captures native contacts across many protein families
• Improved contact prediction in proteins: Using pseudolikelihoods to infer Potts models
• Historical roots of gauge invariance
• Benchmarking inverse statistical approaches for protein structure and design with exactly solvable models
• Aligning biological sequences by exploiting residue conservation and coevolution