ESP: Exploiting Symmetry Prior for Multi-Agent Reinforcement Learning

Abstract

Multi-agent reinforcement learning (MARL) has achieved promising results in recent years. However, most existing reinforcement learning methods require a large amount of data for model training. In addition, data-efficient reinforcement learning requires the construction of strong inductive biases, which are ignored in the current MARL approaches. Inspired by the symmetry phenomenon in multi-agent systems, this paper proposes a framework for exploiting prior knowledge by integrating data augmentation and a well-designed consistency loss into the existing MARL methods. In addition, the proposed framework is model-agnostic and can be applied to most of the current MARL algorithms. Experimental tests on multiple challenging tasks demonstrate the effectiveness of the proposed framework. Moreover, the proposed framework is applied to a physical multi-robot testbed to show its superiority.

arXiv commentAccepted by ECAI 2023

References in corpus (22)

• Dota 2 with Large Scale Deep Reinforcement Learning
• Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments
• The Surprising Effectiveness of PPO in Cooperative, Multi-Agent Games
• Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gauges
• Group Equivariant Convolutional Networks
• QMIX: Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learning
• Reinforcement Learning with Augmented Data
• CURL: Contrastive Unsupervised Representations for Reinforcement Learning
• Decentralized Multi-Agent Pursuit using Deep Reinforcement Learning
• Image Augmentation Is All You Need: Regularizing Deep Reinforcement Learning from Pixels
• A Physics-Informed Deep Learning Paradigm for Traffic State and Fundamental Diagram Estimation
• Isaac Gym: High Performance GPU-Based Physics Simulation For Robot Learning
• MDP Homomorphic Networks: Group Symmetries in Reinforcement Learning
• Scale-Equivariant Steerable Networks
• Invariant Transform Experience Replay: Data Augmentation for Deep Reinforcement Learning
• Graph Convolutional Reinforcement Learning
• Steerable CNNs
• Multi-Agent Game Abstraction via Graph Attention Neural Network
• A Comprehensive Survey of Data Augmentation in Visual Reinforcement Learning
• $\mathrm{SO}(2)$-Equivariant Reinforcement Learning
• Multi-Agent MDP Homomorphic Networks
• Implicit Equivariance in Convolutional Networks

Cited by in corpus (1)

• SIGMA: Sheaf-Informed Geometric Multi-Agent Pathfinding