Learning to Cooperate via Policy Search

Abstract

Cooperative games are those in which both agents share the same payoff structure. Value-based reinforcement-learning algorithms, such as variants of Q-learning, have been applied to learning cooperative games, but they only apply when the game state is completely observable to both agents. Policy search methods are a reasonable alternative to value-based methods for partially observable environments. In this paper, we provide a gradient-based distributed policy-search method for cooperative games and compare the notion of local optimum to that of Nash equilibrium. We demonstrate the effectiveness of this method experimentally in a small, partially observable simulated soccer domain.

Appears in Proceedings of the Sixteenth Conference on Uncertainty in Artificial Intelligence (UAI2000)

References in corpus (3)

• The Complexity of Decentralized Control of Markov Decision Processes
• Learning Finite-State Controllers for Partially Observable Environments
• Learning Policies with External Memory

Cited by in corpus (24)

• The Communicative Multiagent Team Decision Problem: Analyzing Teamwork Theories and Models
• The Complexity of Decentralized Control of Markov Decision Processes
• Decentralized Control of Cooperative Systems: Categorization and Complexity Analysis
• Deep Decentralized Multi-task Multi-Agent Reinforcement Learning under Partial Observability
• MAA*: A Heuristic Search Algorithm for Solving Decentralized POMDPs
• Policy Iteration for Decentralized Control of Markov Decision Processes
• Multiagent Soft Q-Learning
• Learning from Scarce Experience
• Rollout Sampling Policy Iteration for Decentralized POMDPs
• Policy Gradient With Value Function Approximation For Collective Multiagent Planning
• Difference Rewards Policy Gradients
• Reinforcement Learning for Heterogeneous Teams with PALO Bounds
• Proactive Resource Management for LTE in Unlicensed Spectrum: A Deep Learning Perspective
• Reinforcement Learning via Parametric Cost Function Approximation for Multistage Stochastic Programming
• Stochastic Optimization with Parametric Cost Function Approximations
• Scalable Planning and Learning for Multiagent POMDPs: Extended Version
• Distributed Policy Learning Based Random Access for Diversified QoS Requirements
• Exploiting Fast Decaying and Locality in Multi-Agent MDP with Tree Dependence Structure
• Deep Q-Network Based Multi-agent Reinforcement Learning with Binary Action Agents
• Predictive Wireless Based Status Update for Communication-Agnostic Sampling
• Deep Neural Networks Are Congestion Games: From Loss Landscape to Wardrop Equilibrium and Beyond
• Combining Propositional Logic Based Decision Diagrams with Decision Making in Urban Systems
• Revealing Much While Saying Less: Predictive Wireless for Status Update
• Bounds on sample size for policy evaluation in Markov environments