9 papers · 1 filter
Variance-Aware Fine-Grained Gap-Dependent Bounds for Online Reinforcement Learning
Haochen Zhang, Lingzhou Xue, Zhong Zheng
We study model-free online reinforcement learning (RL) for episodic tabular Markov decision processes, focusing on both gap-dependent regret and policy switching cost. While fine-g…
Exact Fast Batch Simulation for Tabular Reinforcement Learning
Haochen Zhang, Lingzhou Xue, Zhong Zheng
Simulation is a fundamental computational primitive in reinforcement learning (RL), yet conventional simulation explicitly generates individual trajectories even when downstream pr…
Understanding Gap-Dependent Regret for Optimism-Based Reinforcement Learning with Linear Function Approximation
Haochen Zhang, Zhong Zheng, Lingzhou Xue
We study gap-dependent regret for reinforcement learning with linear function approximation. While prior works have established gap-dependent guarantees in this setting, existing a…
Q-Learning with Fine-Grained Gap-Dependent Regret
Haochen Zhang, Zhong Zheng, Lingzhou Xue
We study fine-grained gap-dependent regret bounds for model-free reinforcement learning in episodic tabular Markov Decision Processes. Existing model-free algorithms achieve minima…
Regret-Optimal Q-Learning with Low Cost for Single-Agent and Federated Reinforcement Learning
Haochen Zhang, Zhong Zheng, Lingzhou Xue
Motivated by real-world settings where data collection and policy deployment -- whether for a single agent or across multiple agents -- are costly, we study the problem of on-polic…
Gap-Dependent Bounds for Federated -learning
Haochen Zhang, Zhong Zheng, Lingzhou Xue
We present the first gap-dependent analysis of regret and communication cost for on-policy federated -Learning in tabular episodic finite-horizon Markov decision processes (MDPs…
Gap-Dependent Bounds for Q-Learning using Reference-Advantage Decomposition
Zhong Zheng, Haochen Zhang, Lingzhou Xue
We study the gap-dependent bounds of two important algorithms for on-policy Q-learning for finite-horizon episodic tabular Markov Decision Processes (MDPs): UCB-Advantage (Zhang et…
Smoothed Robust Phase Retrieval
Zhong Zheng, Lingzhou Xue
The phase retrieval problem in the presence of noise aims to recover the signal vector of interest from a set of quadratic measurements with infrequent but arbitrary corruptions, a…
Federated Q-Learning with Reference-Advantage Decomposition: Almost Optimal Regret and Logarithmic Communication Cost
Zhong Zheng, Haochen Zhang, Lingzhou Xue
In this paper, we consider model-free federated reinforcement learning for tabular episodic Markov decision processes. Under the coordination of a central server, multiple agents c…