Bayes' Bluff: Opponent Modelling in Poker

Abstract

Poker is a challenging problem for artificial intelligence, with non-deterministic dynamics, partial observability, and the added difficulty of unknown adversaries. Modelling all of the uncertainties in this domain is not an easy task. In this paper we present a Bayesian probabilistic model for a broad class of poker games, separating the uncertainty in the game dynamics from the uncertainty of the opponent's strategy. We then describe approaches to two key subproblems: (i) inferring a posterior over opponent strategies given a prior distribution and observations of their play, and (ii) playing an appropriate response to that distribution. We demonstrate the overall approach on a reduced version of poker using Dirichlet priors and then on the full game of Texas hold'em using a more informed prior. We demonstrate methods for playing effective responses to the opponent, based on the posterior.

Appears in Proceedings of the Twenty-First Conference on Uncertainty in Artificial Intelligence (UAI2005)

Cited by in corpus (27)

• Autonomous Agents Modelling Other Agents: A Comprehensive Survey and Open Problems
• Deep Reinforcement Learning from Self-Play in Imperfect-Information Games
• Opponent Modeling in Deep Reinforcement Learning
• Belief and Truth in Hypothesised Behaviours
• A Generalized Training Approach for Multiagent Learning
• No-Regret Learning Dynamics for Extensive-Form Correlated Equilibrium
• Single Deep Counterfactual Regret Minimization
• Neural Replicator Dynamics
• DREAM: Deep Regret minimization with Advantage baselines and Model-free learning
• Efficient Deviation Types and Learning for Hindsight Rationality in Extensive-Form Games
• Reasoning about Hypothetical Agent Behaviours and their Parameters
• Model-free Neural Counterfactual Regret Minimization with Bootstrap Learning
• Smoothing Method for Approximate Extensive-Form Perfect Equilibrium
• Teaching Social Behavior through Human Reinforcement for Ad hoc Teamwork -The STAR Framework
• Faster Algorithms for Optimal Ex-Ante Coordinated Collusive Strategies in Extensive-Form Zero-Sum Games
• Sparsified Linear Programming for Zero-Sum Equilibrium Finding
• Review, Analysis and Design of a Comprehensive Deep Reinforcement Learning Framework
• Low-Variance and Zero-Variance Baselines for Extensive-Form Games
• Robust Stochastic Bayesian Games for Behavior Space Coverage
• Reduced Space and Faster Convergence in Imperfect-Information Games via Regret-Based Pruning
• Better Regularization for Sequential Decision Spaces: Fast Convergence Rates for Nash, Correlated, and Team Equilibria
• Equivalence Analysis between Counterfactual Regret Minimization and Online Mirror Descent
• Mixture of Public and Private Distributions in Imperfect Information Games
• Search in Imperfect Information Games
• ScrofaZero: Mastering Trick-taking Poker Game Gongzhu by Deep Reinforcement Learning
• Efficient Decentralized Learning Dynamics for Extensive-Form Coarse Correlated Equilibrium: No Expensive Computation of Stationary Distributions Required
• Subgame solving without common knowledge