Statistical Estimation of Confounded Linear MDPs: An Instrumental Variable Approach

Miao Lu, Wenhao Yang, Liangyu Zhang +1

In an Markov decision process (MDP), unobservable confounders may exist and have impacts on the data generating process, so that the classic off-policy evaluation (OPE) estimators…