Reinforcement Learning, Bit by Bit
arXiv:2103.04047
Abstract
Reinforcement learning agents have demonstrated remarkable achievements in simulated environments. Data efficiency poses an impediment to carrying this success over to real environments. The design of data-efficient agents calls for a deeper understanding of information acquisition and representation. We discuss concepts and regret analysis that together offer principled guidance. This line of thinking sheds light on questions of what information to seek, how to seek that information, and what information to retain. To illustrate concepts, we design simple agents that build on them and present computational results that highlight data efficiency.
References in corpus (23)
- Playing Atari with Deep Reinforcement Learning
- #Exploration: A Study of Count-Based Exploration for Deep Reinforcement Learning
- (More) Efficient Reinforcement Learning via Posterior Sampling
- Hybrid Reward Architecture for Reinforcement Learning
- Deep Exploration via Randomized Value Functions
- An Information-Theoretic Approach to Minimax Regret in Partial Monitoring
- Discovery of Useful Questions as Auxiliary Tasks
- An Information-Theoretic Analysis for Thompson Sampling with Many Actions
- Information Directed Sampling for Stochastic Bandits with Graph Feedback
- Variational Bayesian Reinforcement Learning with Regret Bounds
- Information-Theoretic Confidence Bounds for Reinforcement Learning
- Bayesian Adaptive Data Analysis Guarantees from Subgaussianity
- Asymptotically Optimal Information-Directed Sampling
- Information-Directed Exploration for Deep Reinforcement Learning
- Hypermodels for Exploration
- Mirror Descent and the Information Ratio
- Connections Between Mirror Descent, Thompson Sampling and the Information Ratio
- Temporal Difference Uncertainties as a Signal for Exploration
- Information Directed Sampling for Linear Partial Monitoring
- First-Order Bayesian Regret Analysis of Thompson Sampling
- A Bit Better? Quantifying Information for Bandit Learning
- On the Performance of Thompson Sampling on Logistic Bandits
- Deciding What to Learn: A Rate-Distortion Approach