78 citations · 319 across the 37 of their papers we have counts for
19 papers · 1 filter
Why Adam Can Beat SGD: Second-Moment Normalization Yields Sharper Tails
Ruinan Jin, Yingbin Liang, Shaofeng Zou
Despite Adam demonstrating faster empirical convergence than SGD in many applications, much of the existing theory yields guarantees essentially comparable to those of SGD, leaving…
Robust Multi-Agent Reinforcement Learning with State Uncertainty
Sihong He, Songyang Han, Sanbao Su +3
In real-world multi-agent reinforcement learning (MARL) applications, agents may not have perfect state information (e.g., due to inaccurate measurement or malicious attacks), whic…
Model-Free Robust Average-Reward Reinforcement Learning
Yue Wang, Alvaro Velasquez, George Atia +2
Robust Markov decision processes (MDPs) address the challenge of model uncertainty by optimizing the worst-case performance over an uncertainty set of MDPs. In this paper, we focus…
Achieving the Asymptotically Optimal Sample Complexity of Offline Reinforcement Learning: A DRO-Based Approach
Yue Wang, Jinjun Xiong, Shaofeng Zou
Offline reinforcement learning aims to learn from pre-collected datasets without active exploration. This problem faces significant challenges, including limited data availability…
Robust Average-Reward Markov Decision Processes
Yue Wang, Alvaro Velasquez, George Atia +2
In robust Markov decision processes (MDPs), the uncertainty in the transition kernel is addressed by finding a policy that optimizes the worst-case performance over an uncertainty…
Robust Constrained Reinforcement Learning
Yue Wang, Fei Miao, Shaofeng Zou
Constrained reinforcement learning is to maximize the expected reward subject to constraints on utilities/costs. However, the training environment may not be the same as the test o…
Finite-Time Error Bounds for Greedy-GQ
Yue Wang, Yi Zhou, Shaofeng Zou
Greedy-GQ with linear function approximation, originally proposed in \cite{maei2010toward}, is a value-based off-policy algorithm for optimal control in reinforcement learning, and…
Policy Gradient Method For Robust Reinforcement Learning
Yue Wang, Shaofeng Zou
This paper develops the first policy gradient method with global optimality guarantee and complexity analysis for robust reinforcement learning under model mismatch. Robust reinfor…
Online Robust Reinforcement Learning with Model Uncertainty
Yue Wang, Shaofeng Zou
Robust reinforcement learning (RL) is to find a policy that optimizes the worst-case performance over an uncertainty set of MDPs. In this paper, we focus on model-free robust RL, w…
Sample and Communication-Efficient Decentralized Actor-Critic Algorithms with Finite-Time Analysis
Ziyi Chen, Yi Zhou, Rongrong Chen +1
Actor-critic (AC) algorithms have been widely adopted in decentralized multi-agent systems to learn the optimal joint control policy. However, existing decentralized AC algorithms…
Non-Asymptotic Analysis for Two Time-scale TDC with General Smooth Function Approximation
Yue Wang, Shaofeng Zou, Yi Zhou
Temporal-difference learning with gradient correction (TDC) is a two time-scale algorithm for policy evaluation in reinforcement learning. This algorithm was initially proposed wit…
Greedy-GQ with Variance Reduction: Finite-time Analysis and Improved Complexity
Shaocong Ma, Ziyi Chen, Yi Zhou +1
Greedy-GQ is a value-based reinforcement learning (RL) algorithm for optimal control. Recently, the finite-time analysis of Greedy-GQ has been developed under linear function appro…
Learning Graph Neural Networks with Approximate Gradient Descent
Qunwei Li, Shaofeng Zou, Wenliang Zhong
The first provably efficient algorithm for learning graph neural networks (GNNs) with one hidden layer for node information convolution is provided in this paper. Two types of GNNs…
Variance-Reduced Off-Policy TDC Learning: Non-Asymptotic Convergence Analysis
Shaocong Ma, Yi Zhou, Shaofeng Zou
Variance reduction techniques have been successfully applied to temporal-difference (TD) learning and help to improve the sample complexity in policy evaluation. However, the exist…
Finite-sample Analysis of Greedy-GQ with Linear Function Approximation under Markovian Noise
Yue Wang, Shaofeng Zou
Greedy-GQ is an off-policy two timescale algorithm for optimal control in reinforcement learning. This paper develops the first finite-sample analysis for the Greedy-GQ algorithm w…
Two Time-scale Off-Policy TD Learning: Non-asymptotic Analysis over Markovian Samples
Tengyu Xu, Shaofeng Zou, Yingbin Liang
Gradient-based temporal difference (GTD) algorithms are widely used in off-policy learning scenarios. Among them, the two time-scale TD with gradient correction (TDC) algorithm has…
Finite-Sample Analysis for SARSA with Linear Function Approximation
Shaofeng Zou, Tengyu Xu, Yingbin Liang
SARSA is an on-policy algorithm to learn a Markov decision process policy in reinforcement learning. We investigate the SARSA algorithm with linear function approximation under the…
Tightening Mutual Information Based Bounds on Generalization Error
Yuheng Bu, Shaofeng Zou, Venugopal V. Veeravalli
An information-theoretic upper bound on the generalization error of supervised learning algorithms is derived. The bound is constructed in terms of the mutual information between e…
Nonparametric Detection of Anomalous Data Streams
Shaofeng Zou, Yingbin Liang, H. Vincent Poor +1
A nonparametric anomalous hypothesis testing problem is investigated, in which there are totally n sequences with s anomalous sequences to be detected. Each typical sequence contai…