Coordination in Adversarial Sequential Team Games via Multi-Agent Deep Reinforcement Learning
arXiv:1912.07712
Abstract
Many real-world applications involve teams of agents that have to coordinate their actions to reach a common goal against potential adversaries. This paper focuses on zero-sum games where a team of players faces an opponent, as is the case, for example, in Bridge, collusion in poker, and collusion in bidding. The possibility for the team members to communicate before gameplay---that is, coordinate their strategies ex ante---makes the use of behavioral strategies unsatisfactory. We introduce Soft Team Actor-Critic (STAC) as a solution to the team's coordination problem that does not require any prior domain knowledge. STAC allows team members to effectively exploit ex ante communication via exogenous signals that are shared among the team. STAC reaches near-optimal coordinated strategies both in perfectly observable and partially observable games, where previous deep RL algorithms fail to reach optimal coordinated behaviors.
Preliminary version
References in corpus (15)
- Adam: A Method for Stochastic Optimization
- Continuous control with deep reinforcement learning
- Playing Atari with Deep Reinforcement Learning
- Addressing Function Approximation Error in Actor-Critic Methods
- DeepStack: Expert-Level Artificial Intelligence in No-Limit Poker
- Human-level performance in first-person multiplayer games with population-based deep reinforcement learning
- A Survey and Critique of Multiagent Deep Reinforcement Learning
- Learning a Generic Value-Selection Heuristic Inside a Constraint Programming Solver
- Counterfactual Multi-Agent Policy Gradients
- Deep Decentralized Multi-task Multi-Agent Reinforcement Learning under Partial Observability
- Deep Reinforcement Learning from Self-Play in Imperfect-Information Games
- Safe and Nested Subgame Solving for Imperfect-Information Games
- Emergent Coordination Through Competition
- Parameter Sharing Deep Deterministic Policy Gradient for Cooperative Multi-agent Reinforcement Learning
- Deep Counterfactual Regret Minimization