84 citations · 290 across the 36 of their papers we have counts for
12 papers · 1 filter
Off-Policy Evaluation of Ranking Policies under Diverse User Behavior
Haruka Kiyohara, Masatoshi Uehara, Yusuke Narita +3
Ranking interfaces are everywhere in online platforms. There is thus an ever growing interest in their Off-Policy Evaluation (OPE), aiming towards an accurate performance evaluatio…
Minimax Instrumental Variable Regression and Convergence Guarantees without Identification or Closedness
Andrew Bennett, Nathan Kallus, Xiaojie Mao +3
In this paper, we study nonparametric estimation of instrumental variable (IV) regressions. Recently, many flexible machine learning methods have been developed for instrumental va…
A Review of Off-Policy Evaluation in Reinforcement Learning
Masatoshi Uehara, Chengchun Shi, Nathan Kallus
Reinforcement learning (RL) is one of the most vibrant research frontiers in machine learning and has been recently applied to solve a number of challenging problems. In this paper…
Causal Inference Under Unmeasured Confounding With Negative Controls: A Minimax Learning Approach
Nathan Kallus, Xiaojie Mao, Masatoshi Uehara +1
We study the estimation of causal parameters when not all confounders are observed and instead negative controls are available. Recent work has shown how these can enable identific…
Statistically Efficient Off-Policy Policy Gradients
Nathan Kallus, Masatoshi Uehara
Policy gradient methods in reinforcement learning update policy parameters by taking steps in the direction of an estimated gradient of policy value. In this paper, we consider the…
Off-Policy Evaluation and Learning for External Validity under a Covariate Shift
Masahiro Kato, Masatoshi Uehara, Shota Yasui
We consider evaluating and training a new policy for the evaluation data by using the historical data obtained from a different policy. The goal of off-policy evaluation (OPE) is t…
Localized Debiased Machine Learning: Efficient Inference on Quantile Treatment Effects and Beyond
Nathan Kallus, Xiaojie Mao, Masatoshi Uehara
We consider estimating a low-dimensional parameter in an estimating equation involving high-dimensional nuisances that depend on the parameter. A central example is the efficient e…
Efficiently Breaking the Curse of Horizon in Off-Policy Evaluation with Double Reinforcement Learning
Nathan Kallus, Masatoshi Uehara
Off-policy evaluation (OPE) in reinforcement learning is notoriously difficult in long- and infinite-horizon settings due to diminishing overlap between behavior and target policie…
Imputation estimators for unnormalized models with missing data
Masatoshi Uehara, Takeru Matsuda, Jae Kwang Kim
Several statistical models are given in the form of unnormalized densities, and calculation of the normalization constant is intractable. We propose estimation methods for such unn…
Unified estimation framework for unnormalized models with statistical efficiency
Masatoshi Uehara, Takafumi Kanamori, Takashi Takenouchi +1
The parameter estimation of unnormalized models is a challenging problem. The maximum likelihood estimation (MLE) is computationally infeasible for these models since normalizing c…
Analysis of Noise Contrastive Estimation from the Perspective of Asymptotic Variance
Masatoshi Uehara, Takeru Matsuda, Fumiyasu Komaki
There are many models, often called unnormalized models, whose normalizing constants are not calculated in closed form. Maximum likelihood estimation is not directly applicable to…
Generative Adversarial Nets from a Density Ratio Estimation Perspective
Masatoshi Uehara, Issei Sato, Masahiro Suzuki +2
Generative adversarial networks (GANs) are successful deep generative models. GANs are based on a two-player minimax game. However, the objective function derived in the original m…