20 citations · 53 across the 15 of their papers we have counts for
16 papers
When Does On-Policy Interaction Help? Representational Tradeoffs in Value-Based Imitation Learning
Luca Viano, Antoine Moulin, Audrey Huang +3
Imitation learning (IL)---training an agent to replicate expert behavior from demonstrations---underpins applications from robotics to language model training. Standard approaches…
Post-Training at the Edge of Detectability: A Game-Theoretic Approach to Fine-Tuning
Keegan Harris, Brian W. Lee, Ian Waudby-Smith +3
Reinforcement learning (RL) fine-tuning is widely used in language model training to improve performance on a target task while limiting drift from a reference policy. A standard w…
A Unifying View of Coverage in Linear Off-Policy Evaluation
Philip Amortila, Audrey Huang, Akshay Krishnamurthy +1
Off-policy evaluation (OPE) is a fundamental task in reinforcement learning (RL). In the classic setting of linear OPE, finite-sample guarantees often take the form $$ \textrm{Eval…
Model Selection for Off-policy Evaluation: New Algorithms and Experimental Protocol
Pai Liu, Lingfeng Zhao, Shivangi Agarwal +4
Holdout validation and hyperparameter tuning from data is a long-standing problem in offline reinforcement learning (RL). A standard framework is to use off-policy evaluation (OPE)…
Reinforcement Learning under Latent Dynamics: Toward Statistical and Algorithmic Modularity
Philip Amortila, Dylan J. Foster, Nan Jiang +2
Real-world applications of reinforcement learning often involve environments where agents operate on complex, high-dimensional observations, but the underlying (''latent'') dynamic…
Scalable Online Exploration via Coverability
Philip Amortila, Dylan J. Foster, Akshay Krishnamurthy
Exploration is a major challenge in reinforcement learning, especially for high-dimensional domains that require function approximation. We propose exploration objectives -- policy…
Mitigating Covariate Shift in Misspecified Regression with Applications to Reinforcement Learning
Philip Amortila, Tongyi Cao, Akshay Krishnamurthy
A pervasive phenomenon in machine learning applications is distribution shift, where training and deployment conditions for a machine learning model differ. As distribution shift t…
Harnessing Density Ratios for Online Reinforcement Learning
Philip Amortila, Dylan J. Foster, Nan Jiang +2
The theories of offline and online reinforcement learning, despite having evolved in parallel, have begun to show signs of the possibility for a unification, with algorithms and an…
The Optimal Approximation Factors in Misspecified Off-Policy Value Function Estimation
Philip Amortila, Nan Jiang, Csaba Szepesvári
Theoretical guarantees in reinforcement learning (RL) are known to suffer multiplicative blow-up factors with respect to the misspecification error of function approximation. Yet,…
A Few Expert Queries Suffices for Sample-Efficient RL with Resets and Linear Value Approximation
Philip Amortila, Nan Jiang, Dhruv Madeka +1
The current paper studies sample-efficient Reinforcement Learning (RL) in settings where only the optimal value function is assumed to be linearly-realizable. It has recently been…
On Query-efficient Planning in MDPs under Linear Realizability of the Optimal State-value Function
Gellért Weisz, Philip Amortila, Barnabás Janzer +3
We consider local planning in fixed-horizon MDPs with a generative model under the assumption that the optimal value function lies close to the span of a feature map. The generativ…
A Variant of the Wang-Foster-Kakade Lower Bound for the Discounted Setting
Philip Amortila, Nan Jiang, Tengyang Xie
Recently, Wang et al. (2020) showed a highly intriguing hardness result for batch reinforcement learning (RL) with linearly realizable value function and good feature coverage in t…
Exponential Lower Bounds for Planning in MDPs With Linearly-Realizable Optimal Action-Value Functions
Gellért Weisz, Philip Amortila, Csaba Szepesvári
We consider the problem of local planning in fixed-horizon and discounted Markov Decision Processes (MDPs) with linear function approximation and a generative model under the assum…
Constrained Markov Decision Processes via Backward Value Functions
Harsh Satija, Philip Amortila, Joelle Pineau
Although Reinforcement Learning (RL) algorithms have found tremendous success in simulated domains, they often cannot directly be applied to physical systems, especially in cases w…
A Distributional Analysis of Sampling-Based Reinforcement Learning Algorithms
Philip Amortila, Doina Precup, Prakash Panangaden +1
We present a distributional approach to theoretical analyses of reinforcement learning algorithms for constant step-sizes. We demonstrate its effectiveness by presenting simple and…
Learning Graph Weighted Models on Pictures
Philip Amortila, Guillaume Rabusseau
Graph Weighted Models (GWMs) have recently been proposed as a natural generalization of weighted automata over strings and trees to arbitrary families of labeled graphs (and hyperg…