A multi-agent reinforcement learning model of common-pool resource appropriation

Abstract

Humanity faces numerous problems of common-pool resource appropriation. This class of multi-agent social dilemma includes the problems of ensuring sustainable use of fresh water, common fisheries, grazing pastures, and irrigation systems. Abstract models of common-pool resource appropriation based on non-cooperative game theory predict that self-interested agents will generally fail to find socially positive equilibria---a phenomenon called the tragedy of the commons. However, in reality, human societies are sometimes able to discover and implement stable cooperative solutions. Decades of behavioral game theory research have sought to uncover aspects of human behavior that make this possible. Most of that work was based on laboratory experiments where participants only make a single choice: how much to appropriate. Recognizing the importance of spatial and temporal resource dynamics, a recent trend has been toward experiments in more complex real-time video game-like environments. However, standard methods of non-cooperative game theory can no longer be used to generate predictions for this case. Here we show that deep reinforcement learning can be used instead. To that end, we study the emergent behavior of groups of independently learning agents in a partially observed Markov game modeling common-pool resource appropriation. Our experiments highlight the importance of trial-and-error learning in common-pool resource appropriation and shed light on the relationship between exclusion, sustainability, and inequality.

15 pages, 11 figures

Cited by in corpus (23)

• Deep Reinforcement Learning for Multi-Agent Systems: A Review of Challenges, Solutions and Applications
• Deep Reinforcement Learning: An Overview
• Emergent Tool Use From Multi-Agent Autocurricula
• Maintaining cooperation in complex social dilemmas using deep reinforcement learning
• A survey on intrinsic motivation in reinforcement learning
• Evolutionary Reinforcement Learning for Sample-Efficient Multiagent Coordination
• Deep reinforcement learning in World-Earth system models to discover sustainable management strategies
• Learning Multi-Agent Coordination for Enhancing Target Coverage in Directional Sensor Networks
• Learning Reciprocity in Complex Sequential Social Dilemmas
• The Emergence of Adversarial Communication in Multi-Agent Reinforcement Learning
• Consequentialist conditional cooperation in social dilemmas with imperfect information
• Quantifying the effects of environment and population diversity in multi-agent reinforcement learning
• Towards Cooperation in Sequential Prisoner's Dilemmas: a Deep Multiagent Reinforcement Learning Approach
• Model-free conventions in multi-agent reinforcement learning with heterogeneous preferences
• Deception in Social Learning: A Multi-Agent Reinforcement Learning Perspective
• Investigating the Impact of Direct Punishment on the Emergence of Cooperation in Multi-Agent Reinforcement Learning Systems
• A game-theoretic analysis of networked system control for common-pool resource management using multi-agent reinforcement learning
• Stochastic Market Games
• Learning through Probing: a decentralized reinforcement learning architecture for social dilemmas
• Birds of a Feather Flock Together: A Close Look at Cooperation Emergence via Multi-Agent RL
• Improved Cooperation by Exploiting a Common Signal
• Cooperative control of environmental extremes by artificial intelligent agents
• Negotiating Team Formation Using Deep Reinforcement Learning