Conditional Importance Sampling for Off-Policy Learning
arXiv:1910.07479
Abstract
The principal contribution of this paper is a conceptual framework for off-policy reinforcement learning, based on conditional expectations of importance sampling ratios. This framework yields new perspectives and understanding of existing off-policy algorithms, and reveals a broad space of unexplored algorithms. We theoretically analyse this space, and concretely investigate several algorithms that arise from this framework.
arXiv commentAISTATS 2020 camera-ready version
References in corpus (9)
- Continuous control with deep reinforcement learning
- Reinforcement Learning with Unsupervised Auxiliary Tasks
- Distributional Reinforcement Learning with Quantile Regression
- DualDICE: Behavior-Agnostic Estimation of Discounted Stationary Distribution Corrections
- Breaking the Curse of Horizon: Infinite-Horizon Off-Policy Estimation
- Double Reinforcement Learning for Efficient Off-Policy Evaluation in Markov Decision Processes
- Stochastic Variance Reduction Methods for Policy Evaluation
- Consistent On-Line Off-Policy Evaluation
- Policy Optimization via Importance Sampling