98 citations · 333 across the 23 of their papers we have counts for
26 papers
The Update-Equivalence Framework for Decision-Time Planning
Samuel Sokota, Gabriele Farina, David J. Wu +4
The process of revising (or constructing) a policy at execution time -- known as decision-time planning -- has been key to achieving superhuman performance in perfect-information g…
Abstracting Imperfect Information Away from Two-Player Zero-Sum Games
Samuel Sokota, Ryan D'Orazio, Chun Kai Ling +3
In their seminal work, Nayyar et al. (2013) showed that imperfect information can be abstracted away from common-payoff games by having players publicly announce their policies as…
Augmented Thresholds for MONI
César Martínez-Guardiola, Nathaniel K. Brown, Fernando Silva-Coira +3
MONI (Rossi et al., 2022) can store a pangenomic dataset T in small space and later, given a pattern P, quickly find the maximal exact matches (MEMs) of P with respect to T. In thi…
Mastering the Game of No-Press Diplomacy via Human-Regularized Reinforcement Learning and Planning
Anton Bakhtin, David J Wu, Adam Lerer +5
No-press Diplomacy is a complex strategy game involving both cooperation and competition that has served as a benchmark for multi-agent AI research. While self-play reinforcement l…
Human-AI Coordination via Human-Regularized Search and Learning
Hengyuan Hu, David J Wu, Adam Lerer +2
We consider the problem of making AI agents that collaborate well with humans in partially observable fully cooperative environments given datasets of human behavior. Inspired by p…
A Unified Approach to Reinforcement Learning, Quantal Response Equilibria, and Two-Player Zero-Sum Games
Samuel Sokota, Ryan D'Orazio, J. Zico Kolter +5
This work studies an algorithm, which we call magnetic mirror descent, that is inspired by mirror descent and the non-Euclidean proximal gradient algorithm. Our contribution is dem…
Equilibrium Finding in Normal-Form Games Via Greedy Regret Minimization
Hugh Zhang, Adam Lerer, Noam Brown
We extend the classic regret minimization framework for approximating equilibria in normal-form games by greedily weighing iterates based on regrets observed at runtime. Theoretica…
Modeling Strong and Human-Like Gameplay with KL-Regularized Search
Athul Paul Jacob, David J. Wu, Gabriele Farina +5
We consider the task of building strong but human-like policies in multi-agent decision-making problems, given examples of human behavior. Imitation learning is effective at predic…
No-Press Diplomacy from Scratch
Anton Bakhtin, David Wu, Adam Lerer +1
Prior AI successes in complex games have largely focused on settings with at most hundreds of actions at each decision point. In contrast, Diplomacy is a game with more than 10^20…
Scalable Online Planning via Reinforcement Learning Fine-Tuning
Arnaud Fickinger, Hengyuan Hu, Brandon Amos +2
Lookahead search has been a critical component of recent AI successes, such as in the games of chess, go, and poker. However, the search methods used in these games, and in many ot…
Learned Belief Search: Efficiently Improving Policies in Partially Observable Settings
Hengyuan Hu, Adam Lerer, Noam Brown +1
Search is an important tool for computing effective policies in single- and multi-agent environments, and has been crucial for achieving superhuman performance in several benchmark…
Off-Belief Learning
Hengyuan Hu, Adam Lerer, Brandon Cui +4
The standard problem setting in Dec-POMDPs is self-play, where the goal is to find a set of policies that play optimally together. Policies learned through self-play may adopt arbi…
Safe Search for Stackelberg Equilibria in Extensive-Form Games
Chun Kai Ling, Noam Brown
Stackelberg equilibrium is a solution concept in two-player games where the leader has commitment rights over the follower. In recent years, it has become a cornerstone of many sec…
Exploring the acceleration of Nekbone on reconfigurable architectures
Nick Brown
Hardware technological advances are struggling to match scientific ambition, and a key question is how we can use the transistors that we already have more effectively. This is esp…
Human-Level Performance in No-Press Diplomacy via Equilibrium Search
Jonathan Gray, Adam Lerer, Anton Bakhtin +1
Prior AI breakthroughs in complex games have focused on either the purely adversarial or purely cooperative settings. In contrast, Diplomacy is a game of shifting alliances that in…
Unlocking the Potential of Deep Counterfactual Value Networks
Ryan Zarick, Bryan Pellegrino, Noam Brown +1
Deep counterfactual value networks combined with continual resolving provide a way to conduct depth-limited search in imperfect-information games. However, since their introduction…
Combining Deep Reinforcement Learning and Search for Imperfect-Information Games
Noam Brown, Anton Bakhtin, Adam Lerer +1
The combination of deep reinforcement learning and search at both training and test time is a powerful paradigm that has led to a number of successes in single-agent settings and p…
DREAM: Deep Regret minimization with Advantage baselines and Model-free learning
Eric Steinberger, Adam Lerer, Noam Brown
We introduce DREAM, a deep reinforcement learning algorithm that finds optimal strategies in imperfect-information games with multiple agents. Formally, DREAM converges to a Nash E…
Improving Policies via Search in Cooperative Partially Observable Games
Adam Lerer, Hengyuan Hu, Jakob Foerster +1
Recent superhuman results in games have largely been achieved in a variety of zero-sum settings, such as Go and Poker, in which agents need to compete against others. However, just…
Stable-Predictive Optimistic Counterfactual Regret Minimization
Gabriele Farina, Christian Kroer, Noam Brown +1
The CFR framework has been a powerful tool for solving large-scale extensive-form games in practice. However, the theoretical rate at which past CFR-based algorithms converge to th…
Deep Counterfactual Regret Minimization
Noam Brown, Adam Lerer, Sam Gross +1
Counterfactual Regret Minimization (CFR) is the leading framework for solving large imperfect-information games. It converges to an equilibrium by iteratively traversing the game t…
Solving Imperfect-Information Games via Discounted Regret Minimization
Noam Brown, Tuomas Sandholm
Counterfactual regret minimization (CFR) is a family of iterative algorithms that are the most popular and, in practice, fastest approach to approximately solving large imperfect-i…
Depth-Limited Solving for Imperfect-Information Games
Noam Brown, Tuomas Sandholm, Brandon Amos
A fundamental challenge in imperfect-information games is that states do not have well-defined values. As a result, depth-limited search algorithms used in single-agent settings an…
Safe and Nested Subgame Solving for Imperfect-Information Games
Noam Brown, Tuomas Sandholm
In imperfect-information games, the optimal strategy in a subgame may depend on the strategy in other, unreached subgames. Thus a subgame cannot be solved in isolation and must ins…