9 citations · 47 across the 24 of their papers we have counts for
24 papers
Near-Optimal Second-Order Guarantees for Model-Based Adversarial Imitation Learning
Shangzhe Li, Dongruo Zhou, Weitong Zhang
We study online adversarial imitation learning (AIL), where an agent learns from offline expert demonstrations and interacts with the environment online without access to rewards.…
CoPS: Empowering LLM Agents with Provable Cross-Task Experience Sharing
Chen Yang, Chenyang Zhao, Quanquan Gu +1
Sequential reasoning in agent systems has been significantly advanced by large language models (LLMs), yet existing approaches face limitations. Reflection-driven reasoning relies…
Uncertainty-Aware Reward-Free Exploration with General Function Approximation
Junkai Zhang, Weitong Zhang, Dongruo Zhou +1
Mastering multiple tasks through exploration and learning in an environment poses a significant challenge in reinforcement learning (RL). Unsupervised RL has been introduced to add…
Nearly Minimax Optimal Regret for Learning Linear Mixture Stochastic Shortest Path
Qiwei Di, Jiafan He, Dongruo Zhou +1
We study the Stochastic Shortest Path (SSP) problem with a linear mixture transition kernel, where an agent repeatedly interacts with a stochastic environment and seeks to reach ce…
Risk Bounds of Accelerated SGD for Overparameterized Linear Regression
Xuheng Li, Yihe Deng, Jingfeng Wu +2
Accelerated stochastic gradient descent (ASGD) is a workhorse in deep learning and often achieves better generalization performance than SGD. However, existing optimization theory…
Variance-Dependent Regret Bounds for Linear Bandits and Reinforcement Learning: Adaptivity and Computational Efficiency
Heyang Zhao, Jiafan He, Dongruo Zhou +2
Recently, several studies (Zhou et al., 2021a; Zhang et al., 2021b; Kim et al., 2021; Zhou and Gu, 2022) have provided variance-dependent regret bounds for linear contextual bandit…
Nearly Minimax Optimal Reinforcement Learning for Linear Markov Decision Processes
Jiafan He, Heyang Zhao, Dongruo Zhou +1
We study reinforcement learning (RL) with linear function approximation. For episodic time-inhomogeneous linear Markov decision processes (linear MDPs) whose transition probability…
Learning Two-Player Mixture Markov Games: Kernel Function Approximation and Correlated Equilibrium
Chris Junchi Li, Dongruo Zhou, Quanquan Gu +1
We consider learning Nash equilibria in two-player zero-sum Markov Games with nonlinear function approximation, where the action-value function is approximated by a function in a R…
Computationally Efficient Horizon-Free Reinforcement Learning for Linear Mixture MDPs
Dongruo Zhou, Quanquan Gu
Recent studies have shown that episodic reinforcement learning (RL) is not more difficult than contextual bandits, even with a long planning horizon and unknown state transitions.…
Nearly Optimal Algorithms for Linear Contextual Bandits with Adversarial Corruptions
Jiafan He, Dongruo Zhou, Tong Zhang +1
We study the linear contextual bandit problem in the presence of adversarial corruption, where the reward at each round is corrupted by an adversary, and the corruption level (i.e.…
Optimal Online Generalized Linear Regression with Stochastic Noise and Its Application to Heteroscedastic Bandits
Heyang Zhao, Dongruo Zhou, Jiafan He +1
We study the problem of online generalized linear regression in the stochastic setting, where the label is generated from a generalized linear model with possibly unbounded additiv…
Learning Neural Contextual Bandits Through Perturbed Rewards
Yiling Jia, Weitong Zhang, Dongruo Zhou +2
Thanks to the power of representation learning, neural contextual bandit algorithms demonstrate remarkable performance improvement against their classical counterparts. But because…
Iterative Teacher-Aware Learning
Luyao Yuan, Dongruo Zhou, Junhong Shen +5
In human pedagogy, teachers and students can interact adaptively to maximize communication efficiency. The teacher adjusts her teaching method for different students, and the stude…
Linear Contextual Bandits with Adversarial Corruptions
Heyang Zhao, Dongruo Zhou, Quanquan Gu
We study the linear contextual bandit problem in the presence of adversarial corruption, where the interaction between the player and a possibly infinite decision set is contaminat…
Faster Perturbed Stochastic Gradient Methods for Finding Local Minima
Zixiang Chen, Dongruo Zhou, Quanquan Gu
Escaping from saddle points and finding local minimum is a central problem in nonconvex optimization. Perturbed gradient methods are perhaps the simplest approach for this problem.…
Reward-Free Model-Based Reinforcement Learning with Linear Function Approximation
Weitong Zhang, Dongruo Zhou, Quanquan Gu
We study the model-based reward-free reinforcement learning with linear function approximation for episodic Markov decision processes (MDPs). In this setting, the agent works in tw…
Pure Exploration in Kernel and Neural Bandits
Yinglun Zhu, Dongruo Zhou, Ruoxi Jiang +3
We study pure exploration in bandits, where the dimension of the feature representation can be much larger than the number of arms. To overcome the curse of dimensionality, we prop…
Variance-Aware Off-Policy Evaluation with Linear Function Approximation
Yifei Min, Tianhao Wang, Dongruo Zhou +1
We study the off-policy evaluation (OPE) problem in reinforcement learning with linear function approximation, which aims to estimate the value function of a target policy based on…
Provably Efficient Representation Selection in Low-rank Markov Decision Processes: From Online to Offline RL
Weitong Zhang, Jiafan He, Dongruo Zhou +2
The success of deep reinforcement learning (DRL) lies in its ability to learn a representation that is well-suited for the exploration and exploitation task. To understand how the…
Uniform-PAC Bounds for Reinforcement Learning with Linear Function Approximation
Jiafan He, Dongruo Zhou, Quanquan Gu
We study reinforcement learning (RL) with linear function approximation. Existing algorithms for this problem only have high-probability regret and/or Probably Approximately Correc…
Near-optimal Policy Optimization Algorithms for Learning Adversarial Linear Mixture MDPs
Jiafan He, Dongruo Zhou, Quanquan Gu
Learning Markov decision processes (MDPs) in the presence of the adversary is a challenging problem in reinforcement learning (RL). In this paper, we study RL in episodic MDPs with…
Almost Optimal Algorithms for Two-player Zero-Sum Linear Mixture Markov Games
Zixiang Chen, Dongruo Zhou, Quanquan Gu
We study reinforcement learning for two-player zero-sum Markov games with simultaneous moves in the finite-horizon setting, where the transition kernel of the underlying Markov gam…
Provably Efficient Reinforcement Learning with Linear Function Approximation Under Adaptivity Constraints
Tianhao Wang, Dongruo Zhou, Quanquan Gu
We study reinforcement learning (RL) with linear function approximation under the adaptivity constraint. We consider two popular limited adaptivity models: the batch learning model…
Nearly Minimax Optimal Reinforcement Learning for Discounted MDPs
Jiafan He, Dongruo Zhou, Quanquan Gu
We study the reinforcement learning problem for discounted Markov Decision Processes (MDPs) under the tabular setting. We propose a model-based algorithm named UCBVI-, which is…