78 citations · 319 across the 37 of their papers we have counts for
37 papers
Why Adam Can Beat SGD: Second-Moment Normalization Yields Sharper Tails
Ruinan Jin, Yingbin Liang, Shaofeng Zou
Despite Adam demonstrating faster empirical convergence than SGD in many applications, much of the existing theory yields guarantees essentially comparable to those of SGD, leaving…
Robust Multi-Agent Reinforcement Learning with State Uncertainty
Sihong He, Songyang Han, Sanbao Su +3
In real-world multi-agent reinforcement learning (MARL) applications, agents may not have perfect state information (e.g., due to inaccurate measurement or malicious attacks), whic…
Model-Free Robust Average-Reward Reinforcement Learning
Yue Wang, Alvaro Velasquez, George Atia +2
Robust Markov decision processes (MDPs) address the challenge of model uncertainty by optimizing the worst-case performance over an uncertainty set of MDPs. In this paper, we focus…
Achieving the Asymptotically Optimal Sample Complexity of Offline Reinforcement Learning: A DRO-Based Approach
Yue Wang, Jinjun Xiong, Shaofeng Zou
Offline reinforcement learning aims to learn from pre-collected datasets without active exploration. This problem faces significant challenges, including limited data availability…
Robust Average-Reward Markov Decision Processes
Yue Wang, Alvaro Velasquez, George Atia +2
In robust Markov decision processes (MDPs), the uncertainty in the transition kernel is addressed by finding a policy that optimizes the worst-case performance over an uncertainty…
Robust Multi-Hypothesis Testing with Moment Constrained Uncertainty Sets
Akshayaa Magesh, Zhongchang Sun, Venugopal V. Veeravalli +1
The problem of robust binary hypothesis testing is studied. Under both hypotheses, the data-generating distributions are assumed to belong to uncertainty sets constructed through m…
Robust Constrained Reinforcement Learning
Yue Wang, Fei Miao, Shaofeng Zou
Constrained reinforcement learning is to maximize the expected reward subject to constraints on utilities/costs. However, the training environment may not be the same as the test o…
Quickest Anomaly Detection in Sensor Networks With Unlabeled Samples
Zhongchang Sun, Shaofeng Zou
The problem of quickest anomaly detection in networks with unlabeled samples is studied. At some unknown time, an anomaly emerges in the network and changes the data-generating dis…
Understanding Information Disclosure from Secure Computation Output: A Study of Average Salary Computation
Alessandro Baccarini, Marina Blanton, Shaofeng Zou
Secure multi-party computation has seen substantial performance improvements in recent years and is being increasingly used in commercial products. While a significant amount of wo…
Finite-Time Error Bounds for Greedy-GQ
Yue Wang, Yi Zhou, Shaofeng Zou
Greedy-GQ with linear function approximation, originally proposed in \cite{maei2010toward}, is a value-based off-policy algorithm for optimal control in reinforcement learning, and…
Policy Gradient Method For Robust Reinforcement Learning
Yue Wang, Shaofeng Zou
This paper develops the first policy gradient method with global optimality guarantee and complexity analysis for robust reinforcement learning under model mismatch. Robust reinfor…
Kernel Robust Hypothesis Testing
Zhongchang Sun, Shaofeng Zou
The problem of robust hypothesis testing is studied, where under the null and the alternative hypotheses, the data-generating distributions are assumed to be in some uncertainty se…
Quickest Change Detection in Anonymous Heterogeneous Sensor Networks
Zhongchang Sun, Shaofeng Zou, Ruizhi Zhang +1
The problem of quickest change detection (QCD) in anonymous heterogeneous sensor networks is studied. There are heterogeneous sensors and a fusion center. The sensors are clust…
Online Robust Reinforcement Learning with Model Uncertainty
Yue Wang, Shaofeng Zou
Robust reinforcement learning (RL) is to find a policy that optimizes the worst-case performance over an uncertainty set of MDPs. In this paper, we focus on model-free robust RL, w…
Faster Algorithm and Sharper Analysis for Constrained Markov Decision Process
Tianjiao Li, Ziwei Guan, Shaofeng Zou +3
The problem of constrained Markov decision process (CMDP) is investigated, where an agent aims to maximize the expected accumulated discounted reward subject to multiple constraint…
Sample and Communication-Efficient Decentralized Actor-Critic Algorithms with Finite-Time Analysis
Ziyi Chen, Yi Zhou, Rongrong Chen +1
Actor-critic (AC) algorithms have been widely adopted in decentralized multi-agent systems to learn the optimal joint control policy. However, existing decentralized AC algorithms…
Sequential (Quickest) Change Detection: Classical Results and New Directions
Liyan Xie, Shaofeng Zou, Yao Xie +1
Online detection of changes in stochastic systems, referred to as sequential change detection or quickest change detection, is an important research topic in statistics, signal pro…
Non-Asymptotic Analysis for Two Time-scale TDC with General Smooth Function Approximation
Yue Wang, Shaofeng Zou, Yi Zhou
Temporal-difference learning with gradient correction (TDC) is a two time-scale algorithm for policy evaluation in reinforcement learning. This algorithm was initially proposed wit…
Greedy-GQ with Variance Reduction: Finite-time Analysis and Improved Complexity
Shaocong Ma, Ziyi Chen, Yi Zhou +1
Greedy-GQ is a value-based reinforcement learning (RL) algorithm for optimal control. Recently, the finite-time analysis of Greedy-GQ has been developed under linear function appro…
Learning Graph Neural Networks with Approximate Gradient Descent
Qunwei Li, Shaofeng Zou, Wenliang Zhong
The first provably efficient algorithm for learning graph neural networks (GNNs) with one hidden layer for node information convolution is provided in this paper. Two types of GNNs…
Variance-Reduced Off-Policy TDC Learning: Non-Asymptotic Convergence Analysis
Shaocong Ma, Yi Zhou, Shaofeng Zou
Variance reduction techniques have been successfully applied to temporal-difference (TD) learning and help to improve the sample complexity in policy evaluation. However, the exist…
Finite-sample Analysis of Greedy-GQ with Linear Function Approximation under Markovian Noise
Yue Wang, Shaofeng Zou
Greedy-GQ is an off-policy two timescale algorithm for optimal control in reinforcement learning. This paper develops the first finite-sample analysis for the Greedy-GQ algorithm w…
A CNN-Based Blind Denoising Method for Endoscopic Images
Shaofeng Zou, Mingzhu Long, Xuyang Wang +3
The quality of images captured by wireless capsule endoscopy (WCE) is key for doctors to diagnose diseases of gastrointestinal (GI) tract. However, there exist many low-quality end…
Two Time-scale Off-Policy TD Learning: Non-asymptotic Analysis over Markovian Samples
Tengyu Xu, Shaofeng Zou, Yingbin Liang
Gradient-based temporal difference (GTD) algorithms are widely used in off-policy learning scenarios. Among them, the two time-scale TD with gradient correction (TDC) algorithm has…