65 citations · 198 across the 14 of their papers we have counts for
14 papers
Policy Finetuning in Reinforcement Learning via Design of Experiments using Offline Data
Ruiqi Zhang, Andrea Zanette
In some applications of reinforcement learning, a dataset of pre-collected experience is already available but it is also possible to acquire some additional online data to help im…
When is Realizability Sufficient for Off-Policy Reinforcement Learning?
Andrea Zanette
Model-free algorithms for reinforcement learning typically require a condition called Bellman completeness in order to successfully operate off-policy with function approximation,…
Stabilizing Q-learning with Linear Architectures for Provably Efficient Learning
Andrea Zanette, Martin J. Wainwright
The -learning algorithm is a simple and widely-used stochastic approximation scheme for reinforcement learning, but the basic protocol can exhibit instability in conjunction wit…
Bellman Residual Orthogonalization for Offline Reinforcement Learning
Andrea Zanette, Martin J. Wainwright
We propose and analyze a reinforcement learning principle that approximates the Bellman equations by enforcing their validity only along an user-defined space of test functions. Fo…
Provable Benefits of Actor-Critic Methods for Offline Reinforcement Learning
Andrea Zanette, Martin J. Wainwright, Emma Brunskill
Actor-critic methods are widely used in offline reinforcement learning practice, but are not so well-understood theoretically. We propose a new offline actor-critic algorithm that…
Design of Experiments for Stochastic Contextual Linear Bandits
Andrea Zanette, Kefan Dong, Jonathan Lee +1
In the stochastic linear contextual bandit setting there exist several minimax procedures for exploration with policies that are reactive to the data being acquired. In practice, t…
Cautiously Optimistic Policy Optimization and Exploration with Linear Function Approximation
Andrea Zanette, Ching-An Cheng, Alekh Agarwal
Policy optimization methods are popular reinforcement learning algorithms, because their incremental and on-policy nature makes them more stable than the value-based counterparts.…
Exponential Lower Bounds for Batch Reinforcement Learning: Batch RL can be Exponentially Harder than Online RL
Andrea Zanette
Several practical applications of reinforcement learning involve an agent learning from past data without the possibility of further exploration. Often these applications require u…
Provably Efficient Reward-Agnostic Navigation with Linear Value Iteration
Andrea Zanette, Alessandro Lazaric, Mykel J. Kochenderfer +1
There has been growing progress on theoretical analyses for provably efficient learning in MDPs with linear function approximation, but much of the existing work has made strong as…
Learning Near Optimal Policies with Low Inherent Bellman Error
Andrea Zanette, Alessandro Lazaric, Mykel Kochenderfer +1
We study the exploration problem with approximate linear action-value functions in episodic reinforcement learning under the notion of low inherent Bellman error, a condition norma…
Problem Dependent Reinforcement Learning Bounds Which Can Identify Bandit Structure in MDPs
Andrea Zanette, Emma Brunskill
In order to make good decision under uncertainty an agent must learn from observations. To do so, two of the most common frameworks are Contextual Bandits and Markov Decision Proce…
Frequentist Regret Bounds for Randomized Least-Squares Value Iteration
Andrea Zanette, David Brandfonbrener, Emma Brunskill +2
We consider the exploration-exploitation dilemma in finite-horizon reinforcement learning (RL). When the state space is large or continuous, traditional tabular approaches are unfe…
Tighter Problem-Dependent Regret Bounds in Reinforcement Learning without Domain Knowledge using Value Function Bounds
Andrea Zanette, Emma Brunskill
Strong worst-case performance bounds for episodic reinforcement learning exist but fortunately in practice RL algorithms perform much better than such bounds would predict. Algorit…
Robust Super-Level Set Estimation using Gaussian Processes
Andrea Zanette, Junzi Zhang, Mykel J. Kochenderfer
This paper focuses on the problem of determining as large a region as possible where a function exceeds a given threshold with high probability. We assume that we only have access…