5 papers
Stochastic Mirror Descent under Iterate-Dependent Markov Noise: Analysis in the Asymptotic and Finite Time Regimes
Anik Kumar Paul, Shalabh Bhatnagar
We study a stochastic optimization problem in which the sampling distribution depends on the decision variable, and the available samples are generated through an iterate-dependent…
Finite-time analysis of Multi-timescale Stochastic Optimization Algorithms
Kaustubh Kartikey, Shalabh Bhatnagar
We present a finite-time analysis of two smoothed functional stochastic approximation algorithms for simulation-based optimization. The first is a two time-scale gradient-based met…
Finite Time Analysis of Constrained Natural Critic-Actor Algorithm with Improved Sample Complexity
Prashansa Panda, Shalabh Bhatnagar
Recent studies have increasingly focused on non-asymptotic convergence analyses for actor-critic (AC) algorithms. One such effort introduced a two-timescale critic-actor algorithm…
Two-Timescale Critic-Actor for Average Reward MDPs with Function Approximation
Prashansa Panda, Shalabh Bhatnagar
Several recent works have focused on carrying out non-asymptotic convergence analyses for AC algorithms. Recently, a two-timescale critic-actor algorithm has been presented for the…
Finite-Time Analysis of Three-Timescale Constrained Actor-Critic and Constrained Natural Actor-Critic Algorithms
Prashansa Panda, Shalabh Bhatnagar
Actor Critic methods have found immense applications on a wide range of Reinforcement Learning tasks especially when the state-action space is large. In this paper, we consider act…