Acquisition of Chess Knowledge in AlphaZero

Abstract

What is learned by sophisticated neural network agents such as AlphaZero? This question is of both scientific and practical interest. If the representations of strong neural networks bear no resemblance to human concepts, our ability to understand faithful explanations of their decisions will be restricted, ultimately limiting what we can achieve with neural network interpretability. In this work we provide evidence that human knowledge is acquired by the AlphaZero neural network as it trains on the game of chess. By probing for a broad range of human chess concepts we show when and where these concepts are represented in the AlphaZero network. We also provide a behavioural analysis focusing on opening play, including qualitative analysis from chess Grandmaster Vladimir Kramnik. Finally, we carry out a preliminary investigation looking at the low-level details of AlphaZero's representations, and make the resulting behavioural and representational analyses available online.

arXiv comment69 pages, 44 figures

References in corpus (36)

• A Unified Approach to Interpreting Model Predictions
• Towards A Rigorous Science of Interpretable Machine Learning
• Axiomatic Attribution for Deep Networks
• Sanity Checks for Saliency Maps
• Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)
• Towards Robust Interpretability with Self-Explaining Neural Networks
• Understanding the Role of Individual Units in a Deep Neural Network
• Concept Whitening for Interpretable Image Recognition
• Hybrid Reward Architecture for Reinforcement Learning
• The (Un)reliability of saliency methods
• Visualizing and Understanding Atari Agents
• Relational Deep Reinforcement Learning
• Towards Deep Symbolic Reinforcement Learning
• Interestingness Elements for Explainable Reinforcement Learning: Understanding Agents' Capabilities and Limitations
• Acquisition of Chess Knowledge in AlphaZero
• Explaining Classifiers with Causal Concept Effect (CaCE)
• Towards Automatic Concept-based Explanations
• Aligning Superhuman AI with Human Behavior: Chess as a Model System
• On the Robustness of Interpretability Methods
• Efficient Saliency Maps for Explainable AI
• DisCoRL: Continual Reinforcement Learning via Policy Distillation
• Explain Your Move: Understanding Agent Actions Using Specific and Relevant Feature Attribution
• A Theory of Usable Information Under Computational Constraints
• Towards falsifiable interpretability research
• Decoupling feature extraction from policy learning: assessing benefits of state representation learning in goal based robotics
• Concept-based model explanations for Electronic Health Records
• Promises and Pitfalls of Black-Box Concept Learning Models
• S-RL Toolbox: Environments, Datasets and Evaluation Metrics for State Representation Learning
• EDUCE: Explaining model Decisions through Unsupervised Concepts Extraction
• Towards Symbolic Reinforcement Learning with Common Sense
• Learning to play the Chess Variant Crazyhouse above World Champion Level with Deep Neural Networks and Human Data
• Assessing Game Balance with AlphaZero: Exploring Alternative Rule Sets in Chess
• Debiasing Concept-based Explanations with Causal Analysis
• DREAM Architecture: a Developmental Approach to Open-Ended Learning in Robotics
• Domain-Level Explainability -- A Challenge for Creating Trust in Superhuman AI Strategies
• SentiMATE: Learning to play Chess through Natural Language Processing

Cited by in corpus (6)

• Adversarial attacks and defenses in explainable artificial intelligence: A survey
• Acquisition of Chess Knowledge in AlphaZero
• Human-like object concept representations emerge naturally in multimodal large language models
• Learning Chess With Language Models and Transformers
• INSPECT: Intrinsic and Systematic Probing Evaluation for Code Transformers
• Birds look like cars: Adversarial analysis of intrinsically interpretable deep learning