937 citations · 8.2k across the 142 of their papers we have counts for
39 papers · 1 filter
Policy evaluation from a single path: Multi-step methods, mixing and mis-specification
Yaqi Duan, Martin J. Wainwright
We study non-parametric estimation of the value function of an infinite-horizon -discounted Markov reward process (MRP) using observations from a single trajectory. We provide n…
Krylov-Bellman boosting: Super-linear policy evaluation in general state spaces
Eric Xia, Martin J. Wainwright
We present and analyze the Krylov-Bellman Boosting (KBB) algorithm for policy evaluation in general state spaces. It alternates between fitting the Bellman residual using non-param…
Instance-Dependent Confidence and Early Stopping for Reinforcement Learning
Koulik Khamaru, Eric Xia, Martin J. Wainwright +1
Various algorithms for reinforcement learning (RL) exhibit dramatic variation in their convergence rates as a function of problem structure. Such problem-dependent behavior is not…
Optimal policy evaluation using kernel-based temporal difference methods
Yaqi Duan, Mengdi Wang, Martin J. Wainwright
We study methods based on reproducing kernel Hilbert spaces for estimating the value function of an infinite-horizon discounted Markov reward process (MRP). We study a regularized…
Instance-optimality in optimal value estimation: Adaptivity via variance-reduced Q-learning
Koulik Khamaru, Eric Xia, Martin J. Wainwright +1
Various algorithms in reinforcement learning exhibit dramatic variability in their convergence rates and ultimate accuracy as a function of the problem structure. Such instance-spe…
Minimax Off-Policy Evaluation for Multi-Armed Bandits
Cong Ma, Banghua Zhu, Jiantao Jiao +1
We study the problem of off-policy evaluation in the multi-armed bandit model with bounded rewards, and develop minimax rate-optimal procedures under three settings. First, when th…