Learning Fair Policies in Decentralized Cooperative Multi-Agent Reinforcement Learning
arXiv:2012.09421
Abstract
We consider the problem of learning fair policies in (deep) cooperative multi-agent reinforcement learning (MARL). We formalize it in a principled way as the problem of optimizing a welfare function that explicitly encodes two important aspects of fairness: efficiency and equity. As a solution method, we propose a novel neural network architecture, which is composed of two sub-networks specifically designed for taking into account the two aspects of fairness. In experiments, we demonstrate the importance of the two sub-networks for fair optimization. Our overall approach is general as it can accommodate any (sub)differentiable welfare function. Therefore, it is compatible with various notions of fairness that have been proposed in the literature (e.g., lexicographic maximin, generalized Gini social welfare function, proportional fairness). Our solution method is generic and can be implemented in various MARL settings: centralized training and decentralized execution, or fully decentralized. Finally, we experimentally validate our approach in various domains and show that it can perform much better than previous methods.
arXiv commentInternational Conference on Machine Learning
References in corpus (13)
- Learning to Communicate with Deep Multi-Agent Reinforcement Learning
- A Survey and Critique of Multiagent Deep Reinforcement Learning
- Counterfactual Multi-Agent Policy Gradients
- Fully Decentralized Multi-Agent Reinforcement Learning with Networked Agents
- From Parity to Preference-based Notions of Fairness in Classification
- Modeling Others using Oneself in Multi-Agent Reinforcement Learning
- Policy Learning for Fairness in Ranking
- Internet Congestion Control via Deep Reinforcement Learning
- Iroko: A Framework to Prototype Reinforcement Learning for Data Center Traffic Control
- Learning Fair Policies in Multiobjective (Deep) Reinforcement Learning with Average and Discounted Rewards
- Stochastic Variance-Reduced Policy Gradient
- Multi-objective Bandits: Optimizing the Generalized Gini Index
- Learning Fairness in Multi-Agent Systems