Deep Decentralized Multi-task Multi-Agent Reinforcement Learning under Partial Observability

Abstract

Many real-world tasks involve multiple agents with partial observability and limited communication. Learning is challenging in these settings due to local viewpoints of agents, which perceive the world as non-stationary due to concurrently-exploring teammates. Approaches that learn specialized policies for individual tasks face problems when applied to the real world: not only do agents have to learn and store distinct policies for each task, but in practice identities of tasks are often non-observable, making these approaches inapplicable. This paper formalizes and addresses the problem of multi-task multi-agent reinforcement learning under partial observability. We introduce a decentralized single-task learning approach that is robust to concurrent interactions of teammates, and present an approach for distilling single-task policies into a unified policy that performs well across multiple related tasks, without explicit provision of task identity.

Accepted to ICML 2017

References in corpus (2)

• Practical recommendations for gradient-based training of deep architectures
• Learning to Cooperate via Policy Search

Cited by in corpus (64)

• Convergence of Edge Computing and Deep Learning: A Comprehensive Survey
• Deep Reinforcement Learning for Multi-Agent Systems: A Review of Challenges, Solutions and Applications
• A Survey and Critique of Multiagent Deep Reinforcement Learning
• Resource Management in Wireless Networks via Multi-Agent Deep Reinforcement Learning
• Heterogeneous Task Offloading and Resource Allocations via Deep Recurrent Reinforcement Learning in Partial Observable Multi-Fog Networks
• Multi-Agent Reinforcement Learning via Double Averaging Primal-Dual Optimization
• IG-RL: Inductive Graph Reinforcement Learning for Massive-Scale Traffic Signal Control
• Federated Deep Reinforcement Learning
• R-MADDPG for Partially Observable Environments and Limited Communication
• Off-Policy Multi-Agent Decomposed Policy Gradients
• Beyond Robustness: A Taxonomy of Approaches towards Resilient Multi-Robot Systems
• CM3: Cooperative Multi-goal Multi-stage Multi-agent Reinforcement Learning
• Hierarchical Deep Multiagent Reinforcement Learning with Temporal Abstraction
• Spatial Intention Maps for Multi-Agent Mobile Manipulation
• A Generalized Training Approach for Multiagent Learning
• A Deep Recurrent Q Network towards Self-adapting Distributed Microservices architecture
• The Emergence of Adversarial Communication in Multi-Agent Reinforcement Learning
• Distributed multi-agent target search and tracking with Gaussian process and reinforcement learning
• Generalized Hindsight for Reinforcement Learning
• A Decentralized Proximal Point-type Method for Saddle Point Problems
• Spectrum Sharing in Vehicular Networks Based on Multi-Agent Reinforcement Learning
• M${}^3$RL: Mind-aware Multi-agent Management Reinforcement Learning
• Multi-Agent Deep Reinforcement Learning for Liquidation Strategy Analysis
• Macro-Action-Based Deep Multi-Agent Reinforcement Learning
• Learning to Communicate and Correct Pose Errors
• Natural Actor-Critic Converges Globally for Hierarchical Linear Quadratic Regulator
• Likelihood Quantile Networks for Coordinating Multi-Agent Reinforcement Learning
• Learning to Track Dynamic Targets in Partially Known Environments
• Graph Neural Networks for Decentralized Multi-Robot Submodular Action Selection
• Continuous Coordination As a Realistic Scenario for Lifelong Learning
• Efficient Ridesharing Dispatch Using Multi-Agent Reinforcement Learning
• Evaluating the progress of Deep Reinforcement Learning in the real world: aligning domain-agnostic and domain-specific research
• Coordination in Adversarial Sequential Team Games via Multi-Agent Deep Reinforcement Learning
• When Multiple Agents Learn to Schedule: A Distributed Radio Resource Management Framework
• Learning Cross-Domain Correspondence for Control with Dynamics Cycle-Consistency
• Stigmergic Independent Reinforcement Learning for Multi-Agent Collaboration
• Offline Decentralized Multi-Agent Reinforcement Learning
• Multi-Agent Coordination in Adversarial Environments through Signal Mediated Strategies
• Reinforcement Learning for Intelligent Healthcare Systems: A Comprehensive Survey
• Hierarchically Decoupled Imitation for Morphological Transfer
• Communication-Efficient Policy Gradient Methods for Distributed Reinforcement Learning
• Distributed Policy Iteration for Scalable Approximation of Cooperative Multi-Agent Policies
• Decentralized Reinforcement Learning for Multi-Target Search and Detection by a Team of Drones
• Learning Multi-Robot Decentralized Macro-Action-Based Policies via a Centralized Q-Net
• Multi-Robot Formation Control Using Reinforcement Learning
• A Communication-Efficient Multi-Agent Actor-Critic Algorithm for Distributed Reinforcement Learning
• Information State Embedding in Partially Observable Cooperative Multi-Agent Reinforcement Learning
• PAC Guarantees for Cooperative Multi-Agent Reinforcement Learning with Restricted Communication
• Deep Q-Network Based Multi-agent Reinforcement Learning with Binary Action Agents
• Fairness in Multi-agent Reinforcement Learning for Stock Trading
• Multi-Agent Informational Learning Processes
• Independent Reinforcement Learning for Weakly Cooperative Multiagent Traffic Control Problem
• Learning Meta Representations for Agents in Multi-Agent Reinforcement Learning
• Intelligent Link Adaptation for Grant-Free Access Cellular Networks: A Distributed Deep Reinforcement Learning Approach
• Individual specialization in multi-task environments with multiagent reinforcement learners
• AB-Mapper: Attention and BicNet Based Multi-agent Path Finding for Dynamic Crowded Environment
• The Gradient Convergence Bound of Federated Multi-Agent Reinforcement Learning with Efficient Communication
• Multi-Agent Deep Reinforcement Learning For Persistent Monitoring With Sensing, Communication, and Localization Constraints
• Optimal Analysis of Method with Batching for Monotone Stochastic Finite-Sum Variational Inequalities
• Universal Policies to Learn Them All
• Voting-Based Multi-Agent Reinforcement Learning for Intelligent IoT
• Safety aware model-based reinforcement learning for optimal control of a class of output-feedback nonlinear systems
• Direct-Search for a Class of Stochastic Min-Max Problems
• Logical Team Q-learning: An approach towards factored policies in cooperative MARL