5 papers
Auto-exploration for online reinforcement learning
Caleb Ju, Guanghui Lan
The exploration-exploitation dilemma in reinforcement learning (RL) is a fundamental challenge to efficient RL algorithms. Existing algorithms for finite state and action discounte…
Value Mirror Descent for Reinforcement Learning
Zhichao Jia, Guanghui Lan
Value iteration-type methods have been extensively studied for computing a nearly optimal value function in reinforcement learning (RL). Under a generative sampling model, these me…
Strongly-polynomial time and validation analysis of policy gradient methods
Caleb Ju, Guanghui Lan
This paper proposes a novel termination criterion, termed the advantage gap function, for finite state and action Markov decision processes (MDP) and reinforcement learning (RL). B…
Actor-Accelerated Policy Dual Averaging for Reinforcement Learning in Continuous Action Spaces
Ji Gao, Caleb Ju, Guanghui Lan +1
Policy Dual Averaging (PDA) offers a principled Policy Mirror Descent (PMD) framework that more naturally admits value function approximation than standard PMD, enabling the use of…
A model-free first-order method for linear quadratic regulator with sampling complexity
Caleb Ju, Georgios Kotsalis, Guanghui Lan
We consider the classic stochastic linear quadratic regulator (LQR) problem under an infinite horizon average stage cost. By leveraging recent policy gradient methods from reinforc…