PAC Reinforcement Learning without Real-World Feedback
arXiv:1909.10449
Abstract
This work studies reinforcement learning in the Sim-to-Real setting, in which an agent is first trained on a number of simulators before being deployed in the real world, with the aim of decreasing the real-world sample complexity requirement. Using a dynamic model known as a rich observation Markov decision process (ROMDP), we formulate a theoretical framework for Sim-to-Real in the situation where feedback in the real world is not available. We establish real-world sample complexity guarantees that are smaller than what is currently known for directly (i.e., without access to simulators) learning a ROMDP with feedback.
References in corpus (6)
- Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
- Contextual Decision Processes with Low Bellman Rank are PAC-Learnable
- Model-based RL in Contextual Decision Processes: PAC bounds and Exponential Improvements over Model-free Approaches
- On Oracle-Efficient PAC RL with Rich Observations
- Uniform Convergence Rate of the Kernel Density Estimator Adaptive to Intrinsic Volume Dimension
- Reinforcement Learning in Rich-Observation MDPs using Spectral Methods