Causal Multi-Agent Reinforcement Learning: Review and Open Problems
arXiv:2111.06721
Abstract
This paper serves to introduce the reader to the field of multi-agent reinforcement learning (MARL) and its intersection with methods from the study of causality. We highlight key challenges in MARL and discuss these in the context of how causal methods may assist in tackling them. We promote moving toward a 'causality first' perspective on MARL. Specifically, we argue that causality can offer improved safety, interpretability, and robustness, while also providing strong theoretical guarantees for emergent behaviour. We discuss potential solutions for common challenges, and use this context to motivate future research directions.
arXiv commentAccepted at Cooperative AI Workshop, NeurIPS 2021
References in corpus (21)
- Learning to Communicate with Deep Multi-Agent Reinforcement Learning
- Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problems
- Counterfactual Multi-Agent Policy Gradients
- Fully Decentralized Multi-Agent Reinforcement Learning with Networked Agents
- Social Influence as Intrinsic Motivation for Multi-Agent Deep Reinforcement Learning
- Real-Time Bidding with Multi-Agent Reinforcement Learning in Display Advertising
- Multi-Objective Multi-Agent Decision Making: A Utility-based Analysis and Survey
- Dealing with Non-Stationarity in Multi-Agent Deep Reinforcement Learning
- Causality for Machine Learning
- Causal Discovery with Reinforcement Learning
- Amortized Causal Discovery: Learning to Infer Causal Graphs from Time-Series Data
- Modeling Others using Oneself in Multi-Agent Reinforcement Learning
- Causal Imitation Learning with Unobserved Confounders
- Causal Reinforcement Learning using Observational and Interventional Data
- Scalable Multi-Agent Reinforcement Learning for Networked Systems with Average Reward
- On Inductive Biases in Deep Reinforcement Learning
- Meta-Reinforcement Learning Robust to Distributional Shift via Model Identification and Experience Relabeling
- Believe What You See: Implicit Constraint Approach for Offline Multi-Agent Reinforcement Learning
- Causal Markov Decision Processes: Learning Good Interventions Efficiently
- A Critical View of the Structural Causal Model
- Reinforcement Learning is not a Causal problem