12 citations · 15 across the 17 of their papers we have counts for
11 papers · 1 filter
Concentration and Mean-Square Bounds for Contractive Stochastic Approximation: A Unified Elementary Approach
Siddharth Chandak
We establish mean-square and concentration bounds for stochastic approximation (SA) with arbitrary norm contractive mappings, under a multiplicative noise model where the noise may…
Policy Gradient Methods for Non-Markovian Reinforcement Learning
Avik Kar, Siddharth Chandak, Rahul Singh +4
We study policy gradient methods for reinforcement learning in non-Markovian decision processes (NMDPs), where observations and rewards depend on the entire interaction history. To…
Heavy-Tailed and Long-Range Dependent Noise in Stochastic Approximation: A Finite-Time Analysis
Siddharth Chandak, Anuj Yadav, Ayfer Ozgur +1
Stochastic approximation (SA) is a fundamental iterative framework with broad applications in reinforcement learning and optimization. Classical analyses typically rely on martinga…
High-Probability Bounds for SGD under the Polyak-Lojasiewicz Condition with Markovian Noise
Avik Kar, Siddharth Chandak, Rahul Singh +3
We present the first uniform-in-time high-probability bound for SGD under the PL condition, where the gradient noise contains both Markovian and martingale difference components. T…
Regret and Sample Complexity of Online Q-Learning via Concentration of Stochastic Approximation with Time-Inhomogeneous Markov Chains
Rahul Singh, Siddharth Chandak, Eric Moulines +2
We present the first regret bound for classical online Q-learning in infinite-horizon discounted Markov decision processes (MDPs), without relying on optimism or bonus terms. We fi…
Finite-Time Bound for Non-Linear Two-Time-Scale Stochastic Approximation
Siddharth Chandak
Two-time-scale stochastic approximation (SA) is an algorithm with coupled iterations which has found broad applications in reinforcement learning, optimization and game control. In…
Finite-Time Bounds for Two-Time-Scale Stochastic Approximation with Arbitrary Norm Contractions and Markovian Noise
Siddharth Chandak, Shaan Ul Haque, Nicholas Bambos
Two-time-scale Stochastic Approximation (SA) is an iterative algorithm with applications in reinforcement learning and optimization. Prior finite time analysis of such algorithms h…
A Concentration Bound for TD(0) with Function Approximation
Siddharth Chandak, Vivek S. Borkar
We derive uniform all-time concentration bound of the type 'for all for some ' for TD(0) with linear function approximation. We work with online TD learning with…
Equilibrium Bandits: Learning Optimal Equilibria of Unknown Dynamics
Siddharth Chandak, Ilai Bistritz, Nicholas Bambos
Consider a decision-maker that can pick one out of actions to control an unknown system, for turns. The actions are interpreted as different configurations or policies. Hol…
A Concentration Bound for LSPE()
Siddharth Chandak, Vivek S. Borkar, Harsh Dolhare
The popular LSPE() algorithm for policy evaluation is revisited to derive a concentration bound that gives high probability performance guarantees from some time on.
Concentration of Contractive Stochastic Approximation and Reinforcement Learning
Siddharth Chandak, Vivek S. Borkar, Parth Dodhia
Using a martingale concentration inequality, concentration bounds `from time on' are derived for stochastic approximation algorithms with contractive maps and both martingale…