Explaining AlphaGo: Interpreting Contextual Effects in Neural Networks
arXiv:1901.02184
Abstract
In this paper, we propose to disentangle and interpret contextual effects that are encoded in a pre-trained deep neural network. We use our method to explain the gaming strategy of the alphaGo Zero model. Unlike previous studies that visualized image appearances corresponding to the network output or a neural activation only from a global perspective, our research aims to clarify how a certain input unit (dimension) collaborates with other units (dimensions) to constitute inference patterns of the neural network and thus contribute to the network output. The analysis of local contextual effects w.r.t. certain input units is of special values in real applications. Explaining the logic of the alphaGo Zero model is a typical application. In experiments, our method successfully disentangled the rationale of each move during the Go game.
References in corpus (10)
- Understanding Neural Networks Through Deep Visualization
- Understanding Black-box Predictions via Influence Functions
- Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps
- Opening the Black Box of Deep Neural Networks via Information
- Not Just a Black Box: Learning Important Features Through Propagating Activation Differences
- Learning how to explain neural networks: PatternNet and PatternAttribution
- Harnessing Deep Neural Networks with Logic Rules
- Towards Interpretable Deep Neural Networks by Leveraging Adversarial Examples
- Explainable Neural Networks based on Additive Index Models
- Unsupervised Learning of Neural Networks to Explain Neural Networks