Unlocking the Potential of Deep Counterfactual Value Networks

Abstract

Deep counterfactual value networks combined with continual resolving provide a way to conduct depth-limited search in imperfect-information games. However, since their introduction in the DeepStack poker AI, deep counterfactual value networks have not seen widespread adoption. In this paper we introduce several improvements to deep counterfactual value networks, as well as counterfactual regret minimization, and analyze the effects of each change. We combined these improvements to create the poker AI Supremus. We show that while a reimplementation of DeepStack loses head-to-head against the strong benchmark agent Slumbot, Supremus successfully beats Slumbot by an extremely large margin and also achieves a lower exploitability than DeepStack against a local best response. Together, these results show that with our key improvements, deep counterfactual value networks can achieve state-of-the-art performance.

arXiv comment11 pages, 6 figures

References in corpus (7)

• Adam: A Method for Stochastic Optimization
• DeepStack: Expert-Level Artificial Intelligence in No-Limit Poker
• Safe and Nested Subgame Solving for Imperfect-Information Games
• Solving Large Imperfect Information Games Using CFR+
• Equilibrium Approximation Quality of Current No-Limit Poker Bots
• Solving Large Sequential Games with the Excessive Gap Technique
• Single Deep Counterfactual Regret Minimization

Cited by in corpus (1)

• Search in Imperfect Information Games