979 citations · 1k across the 15 of their papers we have counts for
9 papers · 1 filter
Statistical Inference for Stochastic Gradient Descent: Beyond Finite Variance
Jose Blanchet, Peter Glynn, Wenhao Yang
Stochastic gradient descent (SGD) is foundational to large-scale statistical learning and stochastic optimization. However, in some modern statistical learning problems, stochastic…
Estimation and Inference in Distributional Reinforcement Learning
Liangyu Zhang, Yang Peng, Jiadong Liang +2
In this paper, we study distributional reinforcement learning from the perspective of statistical efficiency. We investigate distributional policy evaluation, aiming to estimate th…
Robust Markov Decision Processes without Model Estimation
Wenhao Yang, Han Wang, Tadashi Kozuno +2
Robust Markov Decision Processes (MDPs) are receiving much attention in learning a robust policy which is less sensitive to environment changes. There are an increasing number of w…
Statistical Estimation of Confounded Linear MDPs: An Instrumental Variable Approach
Miao Lu, Wenhao Yang, Liangyu Zhang +1
In an Markov decision process (MDP), unobservable confounders may exist and have impacts on the data generating process, so that the classic off-policy evaluation (OPE) estimators…
A Statistical Analysis of Polyak-Ruppert Averaged Q-learning
Xiang Li, Wenhao Yang, Jiadong Liang +2
We study Q-learning with Polyak-Ruppert averaging in a discounted Markov decision process in synchronous and tabular settings. Under a Lipschitz condition, we establish a functiona…
Towards Theoretical Understandings of Robust Markov Decision Processes: Sample Complexity and Asymptotics
Wenhao Yang, Liangyu Zhang, Zhihua Zhang
In this paper, we study the non-asymptotic and asymptotic performances of the optimal robust policy and value function of robust Markov Decision Processes(MDPs), where the optimal…
Communication-Efficient Local Decentralized SGD Methods
Xiang Li, Wenhao Yang, Shusen Wang +1
Recently, the technique of local updates is a powerful tool in centralized settings to improve communication efficiency via periodical communication. For decentralized settings, it…
On the Convergence of FedAvg on Non-IID Data
Xiang Li, Kaixuan Huang, Wenhao Yang +2
Federated learning enables a large amount of edge computing devices to jointly learn a model without data sharing. As a leading algorithm in this setting, Federated Averaging (\tex…
A Regularized Approach to Sparse Optimal Policy in Reinforcement Learning
Xiang Li, Wenhao Yang, Zhihua Zhang
We propose and study a general framework for regularized Markov decision processes (MDPs) where the goal is to find an optimal policy that maximizes the expected discounted total r…