Worst-Case Regret Bounds for Exploration via Randomized Value Functions
arXiv:1906.02870
Abstract
This paper studies a recent proposal to use randomized value functions to drive exploration in reinforcement learning. These randomized value functions are generated by injecting random noise into the training data, making the approach compatible with many popular methods for estimating parameterized value functions. By providing a worst-case regret bound for tabular finite-horizon Markov decision processes, we show that planning with respect to these randomized value functions can induce provably efficient exploration.
References in corpus (10)
- Noisy Networks for Exploration
- Thompson Sampling for Contextual Bandits with Linear Payoffs
- Exploration by Random Network Distillation
- (More) Efficient Reinforcement Learning via Posterior Sampling
- Deep Exploration via Randomized Value Functions
- A Bayesian Sampling Approach to Exploration in Reinforcement Learning
- Unifying PAC and Regret: Uniform PAC Bounds for Episodic Reinforcement Learning
- Sample Complexity of Episodic Fixed-Horizon Reinforcement Learning
- Randomized Value Functions via Multiplicative Normalizing Flows
- Randomised Bayesian Least-Squares Policy Iteration
Cited by in corpus (6)
- Model-Based Reinforcement Learning with Value-Targeted Regression
- Corruption-robust exploration in episodic reinforcement learning
- Frequentist Regret Bounds for Randomized Least-Squares Value Iteration
- On Function Approximation in Reinforcement Learning: Optimism in the Face of Large State Spaces
- MADE: Exploration via Maximizing Deviation from Explored Regions
- Towards Tractable Optimism in Model-Based Reinforcement Learning