Deep Synoptic Monte Carlo Planning in Reconnaissance Blind Chess
arXiv:2110.01810
Abstract
This paper introduces deep synoptic Monte Carlo planning (DSMCP) for large imperfect information games. The algorithm constructs a belief state with an unweighted particle filter and plans via playouts that start at samples drawn from the belief state. The algorithm accounts for uncertainty by performing inference on "synopses," a novel stochastic abstraction of information states. DSMCP is the basis of the program Penumbra, which won the official 2020 reconnaissance blind chess competition versus 33 other programs. This paper also evaluates algorithm variants that incorporate caution, paranoia, and a novel bandit algorithm. Furthermore, it audits the synopsis features used in Penumbra with per-bit saliency statistics.
arXiv commentAccepted to NeurIPS 2021
References in corpus (13)
- Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm
- DeepStack: Expert-Level Artificial Intelligence in No-Limit Poker
- SmoothGrad: removing noise by adding noise
- Sanity Checks for Saliency Maps
- A Benchmark for Interpretability Methods in Deep Neural Networks
- Anytime Point-Based Approximations for Large POMDPs
- Algorithms for multi-armed bandit problems
- Combining Deep Reinforcement Learning and Search for Imperfect-Information Games
- Accelerating Self-Play Learning in Go
- ELF OpenGo: An Analysis and Open Reimplementation of AlphaZero
- Deep Counterfactual Regret Minimization
- To Balance or Not to Balance: A Simple-yet-Effective Approach for Learning with Long-Tailed Distributions
- On the Complexity of Reconnaissance Blind Chess