Provably Efficient Multi-Task Reinforcement Learning with Model Transfer
arXiv:2107.08622
Abstract
We study multi-task reinforcement learning (RL) in tabular episodic Markov decision processes (MDPs). We formulate a heterogeneous multi-player RL problem, in which a group of players concurrently face similar but not necessarily identical MDPs, with a goal of improving their collective performance through inter-player information sharing. We design and analyze an algorithm based on the idea of model transfer, and provide gap-dependent and gap-independent upper and lower bounds that characterize the intrinsic complexity of the problem.
References in corpus (15)
- Fully Decentralized Multi-Agent Reinforcement Learning with Networked Agents
- Lifelong Federated Reinforcement Learning: A Learning Architecture for Navigation in Cloud Robotic Systems
- Empirical Bernstein Bounds and Sample Variance Penalization
- Transfer Learning in Deep Reinforcement Learning: A Survey
- REGAL: A Regularization based Algorithm for Reinforcement Learning in Weakly Communicating MDPs
- Unifying PAC and Regret: Uniform PAC Bounds for Episodic Reinforcement Learning
- Sample Complexity of Episodic Fixed-Horizon Reinforcement Learning
- Federated Deep Reinforcement Learning
- Sample Complexity of Multi-task Reinforcement Learning
- Sharing Knowledge in Multi-Task Deep Reinforcement Learning
- Transfer from Multiple MDPs
- Non-Asymptotic Gap-Dependent Regret Bounds for Tabular MDPs
- Fine-Grained Gap-Dependent Bounds for Tabular MDPs via Adaptive Multi-Step Bootstrap
- A Model-based Approach for Sample-efficient Multi-task Reinforcement Learning
- Provably Efficient Cooperative Multi-Agent Reinforcement Learning with Function Approximation