DREAM: Deep Regret minimization with Advantage baselines and Model-free learning
arXiv:2006.10410
Abstract
We introduce DREAM, a deep reinforcement learning algorithm that finds optimal strategies in imperfect-information games with multiple agents. Formally, DREAM converges to a Nash Equilibrium in two-player zero-sum games and to an extensive-form coarse correlated equilibrium in all other games. Our primary innovation is an effective algorithm that, in contrast to other regret-based deep learning algorithms, does not require access to a perfect simulator of the game to achieve good performance. We show that DREAM empirically achieves state-of-the-art performance among model-free algorithms in popular benchmark games, and is even competitive with algorithms that do use a perfect simulator.
References in corpus (15)
- Adam: A Method for Stochastic Optimization
- Dueling Network Architectures for Deep Reinforcement Learning
- DeepStack: Expert-Level Artificial Intelligence in No-Limit Poker
- Rainbow: Combining Improvements in Deep Reinforcement Learning
- The Hanabi Challenge: A New Frontier for AI Research
- Bayes' Bluff: Opponent Modelling in Poker
- Deep Reinforcement Learning from Self-Play in Imperfect-Information Games
- Actor-Critic Policy Optimization in Partially Observable Multiagent Environments
- Solving Large Imperfect Information Games Using CFR+
- No-Regret Learning Dynamics for Extensive-Form Correlated Equilibrium
- Equilibrium Approximation Quality of Current No-Limit Poker Bots
- From Poincaré Recurrence to Convergence in Imperfect Information Games: Finding Equilibrium via Regularization
- Single Deep Counterfactual Regret Minimization
- Neural Replicator Dynamics
- Low-Variance and Zero-Variance Baselines for Extensive-Form Games
Cited by in corpus (9)
- Pipeline PSRO: A Scalable Approach for Finding Approximate Nash Equilibria in Large Games
- Efficient Deviation Types and Learning for Hindsight Rationality in Extensive-Form Games
- The Advantage Regret-Matching Actor-Critic
- Model-free Neural Counterfactual Regret Minimization with Bootstrap Learning
- XDO: A Double Oracle Algorithm for Extensive-Form Games
- CFR-MIX: Solving Imperfect Information Extensive-Form Games with Combinatorial Action Space
- Search in Imperfect Information Games
- Learning to Be Cautious
- Adversarial Deep Learning for Online Resource Allocation