14 papers
Variance-Aware Fine-Grained Gap-Dependent Bounds for Online Reinforcement Learning
Haochen Zhang, Lingzhou Xue, Zhong Zheng
We study model-free online reinforcement learning (RL) for episodic tabular Markov decision processes, focusing on both gap-dependent regret and policy switching cost. While fine-g…
Exact Fast Batch Simulation for Tabular Reinforcement Learning
Haochen Zhang, Lingzhou Xue, Zhong Zheng
Simulation is a fundamental computational primitive in reinforcement learning (RL), yet conventional simulation explicitly generates individual trajectories even when downstream pr…
Understanding Gap-Dependent Regret for Optimism-Based Reinforcement Learning with Linear Function Approximation
Haochen Zhang, Zhong Zheng, Lingzhou Xue
We study gap-dependent regret for reinforcement learning with linear function approximation. While prior works have established gap-dependent guarantees in this setting, existing a…
Robotic Video World Models: A Survey of Applications, Research Challenges, Future Directions
Zhiting Mei, Tenny Yin, Ola Shorinwa +9
Video world models have emerged as promising candidates for high-fidelity world models, offering the potential to synthesize high-quality videos capturing fine-grained interactions…
Collapsing Categories for Regression with Mixed Predictors
Chaegeun Song, Zhong Zheng, Bing Li +1
Categorical predictors are omnipresent in everyday regression practice: in fact, most regression data involve some categorical predictors, and this tendency is increasing in modern…
Q-Learning with Fine-Grained Gap-Dependent Regret
Haochen Zhang, Zhong Zheng, Lingzhou Xue
We study fine-grained gap-dependent regret bounds for model-free reinforcement learning in episodic tabular Markov Decision Processes. Existing model-free algorithms achieve minima…
A New Inexact Manifold Proximal Linear Algorithm with Adaptive Stopping Criteria
Zhong Zheng, Xin Yu, Shiqian Ma +1
This paper proposes a new inexact manifold proximal linear (IManPL) algorithm for solving nonsmooth, nonconvex composite optimization problems over an embedded submanifold. At each…
Regret-Optimal Q-Learning with Low Cost for Single-Agent and Federated Reinforcement Learning
Haochen Zhang, Zhong Zheng, Lingzhou Xue
Motivated by real-world settings where data collection and policy deployment -- whether for a single agent or across multiple agents -- are costly, we study the problem of on-polic…
Gap-Dependent Bounds for Federated -learning
Haochen Zhang, Zhong Zheng, Lingzhou Xue
We present the first gap-dependent analysis of regret and communication cost for on-policy federated -Learning in tabular episodic finite-horizon Markov decision processes (MDPs…
Gap-Dependent Bounds for Q-Learning using Reference-Advantage Decomposition
Zhong Zheng, Haochen Zhang, Lingzhou Xue
We study the gap-dependent bounds of two important algorithms for on-policy Q-learning for finite-horizon episodic tabular Markov Decision Processes (MDPs): UCB-Advantage (Zhang et…
Smoothed Robust Phase Retrieval
Zhong Zheng, Lingzhou Xue
The phase retrieval problem in the presence of noise aims to recover the signal vector of interest from a set of quadratic measurements with infrequent but arbitrary corruptions, a…
Adaptive Algorithms for Robust Phase Retrieval
Zhong Zheng, Necdet Serhat Aybat, Shiqian Ma +1
This paper considers robust phase retrieval, which can be cast as a nonsmooth and nonconvex optimization problem. We propose two first-order algorithms with adaptive step sizes: th…
Federated Q-Learning with Reference-Advantage Decomposition: Almost Optimal Regret and Logarithmic Communication Cost
Zhong Zheng, Haochen Zhang, Lingzhou Xue
In this paper, we consider model-free federated reinforcement learning for tabular episodic Markov decision processes. Under the coordination of a central server, multiple agents c…
Federated Q-Learning: Linear Regret Speedup with Low Communication Cost
Zhong Zheng, Fengyu Gao, Lingzhou Xue +1
In this paper, we consider federated reinforcement learning for tabular episodic Markov Decision Processes (MDP) where, under the coordination of a central server, multiple agents…