83 citations · 186 across the 17 of their papers we have counts for
17 papers
TD Convergence: An Optimization Perspective
Kavosh Asadi, Shoham Sabach, Yao Liu +2
We study the convergence behavior of the celebrated temporal-difference (TD) learning algorithm. By looking at the algorithm through the lens of optimization, we first argue that T…
Decision-Focused Model-based Reinforcement Learning for Reward Transfer
Abhishek Sharma, Sonali Parbhoo, Omer Gottesman +1
Model-based reinforcement learning (MBRL) provides a way to learn a transition model of the environment, which can then be used to plan personalized policies for different patient…
On the Geometry of Reinforcement Learning in Continuous State and Action Spaces
Saket Tiwari, Omer Gottesman, George Konidaris
Advances in reinforcement learning have led to its successful application in complex tasks with continuous state and action spaces. Despite these advances in practice, most theoret…
A Bayesian Approach to Learning Bandit Structure in Markov Decision Processes
Kelly W. Zhang, Omer Gottesman, Finale Doshi-Velez
In the reinforcement learning literature, there are many algorithms developed for either Contextual Bandit (CB) or Markov Decision Processes (MDP) environments. However, when deplo…
Faster Deep Reinforcement Learning with Slower Online Network
Kavosh Asadi, Rasool Fakoor, Omer Gottesman +3
Deep reinforcement learning algorithms often use two networks for value function optimization: an online network, and a target network that tracks the online network with some dela…
Identification of Subgroups With Similar Benefits in Off-Policy Policy Evaluation
Ramtin Keramati, Omer Gottesman, Leo Anthony Celi +2
Off-policy policy evaluation methods for sequential decision making can be used to help identify if a proposed decision policy is better than a current baseline policy. However, a…
Coarse-Grained Smoothness for RL in Metric Spaces
Omer Gottesman, Kavosh Asadi, Cameron Allen +3
Principled decision-making in continuous state--action spaces is impossible without some assumptions. A common approach is to assume Lipschitz continuity of the Q-function. We show…
State Relevance for Off-Policy Evaluation
Simon P. Shen, Yecheng Jason Ma, Omer Gottesman +1
Importance sampling-based estimators for off-policy evaluation (OPE) are valued for their simplicity, unbiasedness, and reliance on relatively few assumptions. However, the varianc…
Learning to search efficiently for causally near-optimal treatments
Samuel Håkansson, Viktor Lindblom, Omer Gottesman +1
Finding an effective medical treatment often requires a search by trial and error. Making this search more efficient by minimizing the number of unnecessary trials could lower both…
Interpretable Off-Policy Evaluation in Reinforcement Learning by Highlighting Influential Transitions
Omer Gottesman, Joseph Futoma, Yao Liu +4
Off-policy evaluation in reinforcement learning offers the chance of using observational data to improve future outcomes in domains such as healthcare and education, but safe deplo…
A general method for regularizing tensor decomposition methods via pseudo-data
Omer Gottesman, Weiwei Pan, Finale Doshi-Velez
Tensor decomposition methods allow us to learn the parameters of latent variable models through decomposition of low-order moments of data. A significant limitation of these algori…
Combining Parametric and Nonparametric Models for Off-Policy Evaluation
Omer Gottesman, Yao Liu, Scott Sussex +2
We consider a model-based approach to perform batch off-policy evaluation in reinforcement learning. Our method takes a mixture-of-experts approach to combine parametric and non-pa…
Improving Sepsis Treatment Strategies by Combining Deep and Kernel-Based Reinforcement Learning
Xuefeng Peng, Yi Ding, David Wihl +6
Sepsis is the leading cause of mortality in the ICU. It is challenging to manage because individual patients respond differently to treatment. Thus, tailoring treatment to the indi…
Behaviour Policy Estimation in Off-Policy Policy Evaluation: Calibration Matters
Aniruddh Raghu, Omer Gottesman, Yao Liu +4
In this work, we consider the problem of estimating a behaviour policy for use in Off-Policy Policy Evaluation (OPE) when the true behaviour policy is unknown. Via a series of empi…
Evaluating Reinforcement Learning Algorithms in Observational Health Settings
Omer Gottesman, Fredrik Johansson, Joshua Meier +16
Much attention has been devoted recently to the development of machine learning algorithms with the goal of improving treatment policies in healthcare. Reinforcement learning (RL)…
Representation Balancing MDPs for Off-Policy Policy Evaluation
Yao Liu, Omer Gottesman, Aniruddh Raghu +4
We study the problem of off-policy policy evaluation (OPPE) in RL. In contrast to prior work, we consider how to estimate both the individual policy value and average policy value…
Weighted Tensor Decomposition for Learning Latent Variables with Partial Data
Omer Gottesman, Weiwei Pan, Finale Doshi-Velez
Tensor decomposition methods are popular tools for learning latent variables given only lower-order moments of the data. However, the standard assumption is that we have sufficient…