979 citations · 1k across the 15 of their papers we have counts for
15 papers
Statistical Inference for Stochastic Gradient Descent: Beyond Finite Variance
Jose Blanchet, Peter Glynn, Wenhao Yang
Stochastic gradient descent (SGD) is foundational to large-scale statistical learning and stochastic optimization. However, in some modern statistical learning problems, stochastic…
Estimation and Inference in Distributional Reinforcement Learning
Liangyu Zhang, Yang Peng, Jiadong Liang +2
In this paper, we study distributional reinforcement learning from the perspective of statistical efficiency. We investigate distributional policy evaluation, aiming to estimate th…
Regularization and Variance-Weighted Regression Achieves Minimax Optimality in Linear MDPs: Theory and Practice
Toshinori Kitamura, Tadashi Kozuno, Yunhao Tang +12
Mirror descent value iteration (MDVI), an abstraction of Kullback-Leibler (KL) and entropy-regularized reinforcement learning (RL), has served as the basis for recent high-performi…
Non-stationary Projection-free Online Learning with Dynamic and Adaptive Regret Guarantees
Yibo Wang, Wenhao Yang, Wei Jiang +5
Projection-free online learning has drawn increasing interest due to its efficiency in solving high-dimensional problems with complicated constraints. However, most existing projec…
Semi-Infinitely Constrained Markov Decision Processes and Efficient Reinforcement Learning
Liangyu Zhang, Yang Peng, Wenhao Yang +1
We propose a novel generalization of constrained Markov decision processes (CMDPs) that we call the \emph{semi-infinitely constrained Markov decision process} (SICMDP). Particularl…
Robust Markov Decision Processes without Model Estimation
Wenhao Yang, Han Wang, Tadashi Kozuno +2
Robust Markov Decision Processes (MDPs) are receiving much attention in learning a robust policy which is less sensitive to environment changes. There are an increasing number of w…
Statistical Estimation of Confounded Linear MDPs: An Instrumental Variable Approach
Miao Lu, Wenhao Yang, Liangyu Zhang +1
In an Markov decision process (MDP), unobservable confounders may exist and have impacts on the data generating process, so that the classic off-policy evaluation (OPE) estimators…
KL-Entropy-Regularized RL with a Generative Model is Minimax Optimal
Tadashi Kozuno, Wenhao Yang, Nino Vieillard +10
In this work, we consider and analyze the sample complexity of model-free reinforcement learning with a generative model. Particularly, we analyze mirror descent value iteration (M…
Federated Reinforcement Learning with Environment Heterogeneity
Hao Jin, Yang Peng, Wenhao Yang +2
We study a Federated Reinforcement Learning (FedRL) problem in which agents collaboratively learn a single policy without sharing the trajectories they collected during agent-e…
A Statistical Analysis of Polyak-Ruppert Averaged Q-learning
Xiang Li, Wenhao Yang, Jiadong Liang +2
We study Q-learning with Polyak-Ruppert averaging in a discounted Markov decision process in synchronous and tabular settings. Under a Lipschitz condition, we establish a functiona…
Towards Theoretical Understandings of Robust Markov Decision Processes: Sample Complexity and Asymptotics
Wenhao Yang, Liangyu Zhang, Zhihua Zhang
In this paper, we study the non-asymptotic and asymptotic performances of the optimal robust policy and value function of robust Markov Decision Processes(MDPs), where the optimal…
Finding the Near Optimal Policy via Adaptive Reduced Regularization in MDPs
Wenhao Yang, Xiang Li, Guangzeng Xie +1
Regularized MDPs serve as a smooth version of original MDPs. However, biased optimal policy always exists for regularized MDPs. Instead of making the coefficientλof regularized ter…
Communication-Efficient Local Decentralized SGD Methods
Xiang Li, Wenhao Yang, Shusen Wang +1
Recently, the technique of local updates is a powerful tool in centralized settings to improve communication efficiency via periodical communication. For decentralized settings, it…
On the Convergence of FedAvg on Non-IID Data
Xiang Li, Kaixuan Huang, Wenhao Yang +2
Federated learning enables a large amount of edge computing devices to jointly learn a model without data sharing. As a leading algorithm in this setting, Federated Averaging (\tex…
A Regularized Approach to Sparse Optimal Policy in Reinforcement Learning
Xiang Li, Wenhao Yang, Zhihua Zhang
We propose and study a general framework for regularized Markov decision processes (MDPs) where the goal is to find an optimal policy that maximizes the expected discounted total r…