A game-theoretic analysis of networked system control for common-pool resource management using multi-agent reinforcement learning

Abstract

Multi-agent reinforcement learning has recently shown great promise as an approach to networked system control. Arguably, one of the most difficult and important tasks for which large scale networked system control is applicable is common-pool resource management. Crucial common-pool resources include arable land, fresh water, wetlands, wildlife, fish stock, forests and the atmosphere, of which proper management is related to some of society's greatest challenges such as food security, inequality and climate change. Here we take inspiration from a recent research program investigating the game-theoretic incentives of humans in social dilemma situations such as the well-known tragedy of the commons. However, instead of focusing on biologically evolved human-like agents, our concern is rather to better understand the learning and operating behaviour of engineered networked systems comprising general-purpose reinforcement learning agents, subject only to nonbiological constraints such as memory, computation and communication bandwidth. Harnessing tools from empirical game-theoretic analysis, we analyse the differences in resulting solution concepts that stem from employing different information structures in the design of networked multi-agent systems. These information structures pertain to the type of information shared between agents as well as the employed communication protocol and network topology. Our analysis contributes new insights into the consequences associated with certain design choices and provides an additional dimension of comparison between systems beyond efficiency, robustness, scalability and mean control performance.

arXiv comment17 pages, 16 Figures, to appear in Advances of Neural Information Processing Systems (NeurIPS) conference, 2020

References in corpus (13)

• Neural Message Passing for Quantum Chemistry
• Learning to Communicate with Deep Multi-Agent Reinforcement Learning
• Stabilising Experience Replay for Deep Multi-Agent Reinforcement Learning
• Emergent Tool Use From Multi-Agent Autocurricula
• Fully Decentralized Multi-Agent Reinforcement Learning with Networked Agents
• Multi-agent Reinforcement Learning in Sequential Social Dilemmas
• Social Influence as Intrinsic Motivation for Multi-Agent Deep Reinforcement Learning
• Maintaining cooperation in complex social dilemmas using deep reinforcement learning
• Inequity aversion improves cooperation in intertemporal social dilemmas
• A multi-agent reinforcement learning model of common-pool resource appropriation
• Autocurricula and the Emergence of Innovation from Social Interaction: A Manifesto for Multi-Agent Intelligence Research
• Multi-agent Reinforcement Learning for Networked System Control
• Consequentialist conditional cooperation in social dilemmas with imperfect information