Active Reinforcement Learning: Observing Rewards at a Cost

Abstract

Active reinforcement learning (ARL) is a variant on reinforcement learning where the agent does not observe the reward unless it chooses to pay a query cost c > 0. The central question of ARL is how to quantify the long-term value of reward information. Even in multi-armed bandits, computing the value of this information is intractable and we have to rely on heuristics. We propose and evaluate several heuristic approaches for ARL in multi-armed bandits and (tabular) Markov decision processes, and discuss and illustrate some challenging aspects of the ARL problem.

Originally appeared at the NeurIPS 2016 "Future of Interactive Learning Machines (FILM)" workshop

References in corpus (5)

• Selecting Computations: Theory and Applications
• A Notation for Markov Decision Processes
• Regret Lower Bound and Optimal Algorithm in Finite Stochastic Partial Monitoring
• Active Learning for Autonomous Intelligent Agents: Exploration, Curiosity, and Interaction
• Learning the Preferences of Ignorant, Inconsistent Agents

Cited by in corpus (4)

• Reinforcement Learning for Selective Key Applications in Power Systems: Recent Advances and Future Challenges
• AI Research Considerations for Human Existential Safety (ARCHES)
• Learning to Request Guidance in Emergent Communication
• Active Offline Policy Selection