4 papers · 1 filter
Regret and Sample Complexity of Online Q-Learning via Concentration of Stochastic Approximation with Time-Inhomogeneous Markov Chains
Rahul Singh, Siddharth Chandak, Eric Moulines +2
We present the first regret bound for classical online Q-learning in infinite-horizon discounted Markov decision processes (MDPs), without relying on optimism or bonus terms. We fi…
A Concentration Bound for TD(0) with Function Approximation
Siddharth Chandak, Vivek S. Borkar
We derive uniform all-time concentration bound of the type 'for all for some ' for TD(0) with linear function approximation. We work with online TD learning with…
An Actor-Critic Algorithm with Function Approximation for Risk Sensitive Cost Markov Decision Processes
Soumyajit Guin, Vivek S. Borkar, Shalabh Bhatnagar
In this paper, we consider the risk-sensitive cost criterion with exponentiated costs for Markov decision processes and develop a model-free policy gradient algorithm in this setti…
Actor-Critic or Critic-Actor? A Tale of Two Time Scales
Shalabh Bhatnagar, Vivek S. Borkar, Soumyajit Guin
We revisit the standard formulation of tabular actor-critic algorithm as a two time-scale stochastic approximation with value function computed on a faster time-scale and policy co…