17 citations · 71 across the 33 of their papers we have counts for
22 papers · 1 filter
Testing for the Markov Property in Time Series via Deep Conditional Generative Learning
Yunzhe Zhou, Chengchun Shi, Lexin Li +1
The Markov property is widely imposed in analysis of time series data. Correspondingly, testing the Markov property, and relatedly, inferring the order of a Markov model, are of pa…
Sequential Knockoffs for Variable Selection in Reinforcement Learning
Tao Ma, Jin Zhu, Hengrui Cai +4
In real-world applications of reinforcement learning, it is often challenging to obtain a state representation that is parsimonious and satisfies the Markov property without prior…
Value Enhancement of Reinforcement Learning via Efficient and Robust Trust Region Optimization
Chengchun Shi, Zhengling Qi, Jianing Wang +1
Reinforcement learning (RL) is a powerful machine learning technique that enables an intelligent agent to learn an optimal policy that maximizes the cumulative rewards in sequentia…
Deep Spectral Q-learning with Application to Mobile Health
Yuhe Gao, Chengchun Shi, Rui Song
Dynamic treatment regimes assign personalized treatments to patients sequentially over time based on their baseline information and time-varying covariates. In mobile health applic…
A Reinforcement Learning Framework for Dynamic Mediation Analysis
Lin Ge, Jitao Wang, Chengchun Shi +2
Mediation analysis learns the causal effect transmitted via mediator variables between treatments and outcomes and receives increasing attention in various scientific domains to el…
Quantile Off-Policy Evaluation via Deep Conditional Generative Learning
Yang Xu, Chengchun Shi, Shikai Luo +2
Off-Policy evaluation (OPE) is concerned with evaluating a new target policy using offline data generated by a potentially different behavior policy. It is critical in a number of…
A Review of Off-Policy Evaluation in Reinforcement Learning
Masatoshi Uehara, Chengchun Shi, Nathan Kallus
Reinforcement learning (RL) is one of the most vibrant research frontiers in machine learning and has been recently applied to solve a number of challenging problems. In this paper…
An Instrumental Variable Approach to Confounded Off-Policy Evaluation
Yang Xu, Jin Zhu, Chengchun Shi +2
Off-policy evaluation (OPE) is a method for estimating the return of a target policy using some pre-collected observational data generated by a potentially different behavior polic…
Doubly Inhomogeneous Reinforcement Learning
Liyuan Hu, Mengbing Li, Chengchun Shi +2
This paper studies reinforcement learning (RL) in doubly inhomogeneous environments under temporal non-stationarity and subject heterogeneity. In a number of applications, it is co…
Optimizing Pessimism in Dynamic Treatment Regimes: A Bayesian Learning Approach
Yunzhe Zhou, Zhengling Qi, Chengchun Shi +1
In this article, we propose a novel pessimism-based Bayesian learning method for optimal dynamic treatment regimes in the offline setting. When the coverage condition does not hold…
Conformal Off-policy Prediction
Yingying Zhang, Chengchun Shi, Shikai Luo
Off-policy evaluation is critical in a number of applications where new policies need to be evaluated offline before online deployment. Most existing methods focus on the expected…
Testing Stationarity and Change Point Detection in Reinforcement Learning
Mengbing Li, Chengchun Shi, Zhenke Wu +1
We consider offline reinforcement learning (RL) methods in possibly nonstationary environments. Many existing RL algorithms in the literature rely on the stationarity assumption th…
Statistically Efficient Advantage Learning for Offline Reinforcement Learning in Infinite Horizons
Chengchun Shi, Shikai Luo, Yuan Le +2
We consider reinforcement learning (RL) methods in offline domains without additional online data collection, such as mobile health applications. Most of existing policy optimizati…
Off-Policy Confidence Interval Estimation with Confounded Markov Decision Process
Chengchun Shi, Jin Zhu, Ye Shen +3
This paper is concerned with constructing a confidence interval for a target policy's value offline based on a pre-collected observational data in infinite horizon settings. Most o…
A Multi-Agent Reinforcement Learning Framework for Off-Policy Evaluation in Two-sided Markets
Chengchun Shi, Runzhe Wan, Ge Song +3
The two-sided markets such as ride-sharing companies often involve a group of subjects who are making sequential decisions across time and/or location. With the rapid development o…
Deeply-Debiased Off-Policy Interval Estimation
Chengchun Shi, Runzhe Wan, Victor Chernozhukov +1
Off-policy evaluation learns a target policy's value with a historical dataset generated by a different behavior policy. In addition to a point estimate, many applications would be…
Testing Directed Acyclic Graph via Structural, Supervised and Generative Adversarial Learning
Chengchun Shi, Yunzhe Zhou, Lexin Li
In this article, we propose a new hypothesis testing method for directed acyclic graph (DAG). While there is a rich class of DAG estimation methods, there is a relative paucity of…
Deep Jump Learning for Off-Policy Evaluation in Continuous Treatment Settings
Hengrui Cai, Chengchun Shi, Rui Song +1
We consider off-policy evaluation (OPE) in continuous treatment settings, such as personalized dose-finding. In OPE, one aims to estimate the mean outcome under a new treatment dec…
Double Generative Adversarial Networks for Conditional Independence Testing
Chengchun Shi, Tianlin Xu, Wicher Bergsma +1
In this article, we study the problem of high-dimensional conditional independence testing, a key building block in statistics and machine learning. We propose an inferential proce…
Does the Markov Decision Process Fit the Data: Testing for the Markov Property in Sequential Decision Making
Chengchun Shi, Runzhe Wan, Rui Song +2
The Markov assumption (MA) is fundamental to the empirical validity of reinforcement learning. In this paper, we propose a novel Forward-Backward Learning procedure to test MA in s…
Statistical Inference of the Value Function for Reinforcement Learning in Infinite Horizon Settings
C. Shi, S. Zhang, W. Lu +1
Reinforcement learning is a general technique that allows an agent to learn an optimal policy and interact with an environment in sequential decision making problems. The goodness…
Robust Learning for Optimal Treatment Decision with NP-Dimensionality
Chengchun Shi, Rui Song, Wenbin Lu
In order to identify important variables that are involved in making optimal treatment decision, Lu et al. (2013) proposed a penalized least squared regression framework for a fixe…