12 citations · 15 across the 17 of their papers we have counts for
22 papers
Safe Learning of Adaptive Control Policies for Remote Patient Monitoring
Ramanan Tamizholi, Siddharth Chandak, Isha Thapa +2
Remote Patient Monitoring (RPM) enables continuous observation of patients in their daily environments, improving both health outcomes and quality of life. A key challenge in RPM i…
Concentration and Mean-Square Bounds for Contractive Stochastic Approximation: A Unified Elementary Approach
Siddharth Chandak
We establish mean-square and concentration bounds for stochastic approximation (SA) with arbitrary norm contractive mappings, under a multiplicative noise model where the noise may…
Policy Gradient Methods for Non-Markovian Reinforcement Learning
Avik Kar, Siddharth Chandak, Rahul Singh +4
We study policy gradient methods for reinforcement learning in non-Markovian decision processes (NMDPs), where observations and rewards depend on the entire interaction history. To…
Last-Iterate Guarantees for Learning in Co-coercive Games
Siddharth Chandak, Ramanan Tamizholi, Nicholas Bambos
We establish finite-time last-iterate guarantees for vanilla stochastic gradient descent in co-coercive games under noisy feedback. This is a broad class of games that is more gene…
Heavy-Tailed and Long-Range Dependent Noise in Stochastic Approximation: A Finite-Time Analysis
Siddharth Chandak, Anuj Yadav, Ayfer Ozgur +1
Stochastic approximation (SA) is a fundamental iterative framework with broad applications in reinforcement learning and optimization. Classical analyses typically rely on martinga…
High-Probability Bounds for SGD under the Polyak-Lojasiewicz Condition with Markovian Noise
Avik Kar, Siddharth Chandak, Rahul Singh +3
We present the first uniform-in-time high-probability bound for SGD under the PL condition, where the gradient noise contains both Markovian and martingale difference components. T…
Regret and Sample Complexity of Online Q-Learning via Concentration of Stochastic Approximation with Time-Inhomogeneous Markov Chains
Rahul Singh, Siddharth Chandak, Eric Moulines +2
We present the first regret bound for classical online Q-learning in infinite-horizon discounted Markov decision processes (MDPs), without relying on optimism or bonus terms. We fi…
Choose Your Battles: Distributed Learning Over Multiple Tug of War Games
Siddharth Chandak, Ilai Bistritz, Nicholas Bambos
Consider players and games taking place simultaneously. Each of these games is modeled as a Tug-of-War (ToW) game where increasing the action of one player decreases the re…
Finite-Time Bound for Non-Linear Two-Time-Scale Stochastic Approximation
Siddharth Chandak
Two-time-scale stochastic approximation (SA) is an algorithm with coupled iterations which has found broad applications in reinforcement learning, optimization and game control. In…
Optimal Control for Remote Patient Monitoring with Multidimensional Health States
Siddharth Chandak, Isha Thapa, Nicholas Bambos +1
Selecting the right monitoring level in Remote Patient Monitoring (RPM) systems for e-healthcare is crucial for balancing patient outcomes, various resources, and patient's quality…
Finite-Time Bounds for Two-Time-Scale Stochastic Approximation with Arbitrary Norm Contractions and Markovian Noise
Siddharth Chandak, Shaan Ul Haque, Nicholas Bambos
Two-time-scale Stochastic Approximation (SA) is an iterative algorithm with applications in reinforcement learning and optimization. Prior finite time analysis of such algorithms h…
Non-Expansive Mappings in Two-Time-Scale Stochastic Approximation: Finite-Time Analysis
Siddharth Chandak
Two-time-scale stochastic approximation algorithms are iterative methods used in applications such as optimization, reinforcement learning, and control. Finite-time analysis of the…
Learning to Control Unknown Strongly Monotone Games
Siddharth Chandak, Ilai Bistritz, Nicholas Bambos
Consider a strongly monotone game where the players' utility functions include a reward function and a linear term for each dimension, with coefficients that are controlled by the…
Tiered Service Architecture for Remote Patient Monitoring
Siddharth Chandak, Isha Thapa, Nicholas Bambos +1
We develop a remote patient monitoring (RPM) service architecture, which has two tiers of monitoring: ordinary and intensive. The patient's health state improves or worsens in each…
A Concentration Bound for TD(0) with Function Approximation
Siddharth Chandak, Vivek S. Borkar
We derive uniform all-time concentration bound of the type 'for all for some ' for TD(0) with linear function approximation. We work with online TD learning with…
Equilibrium Bandits: Learning Optimal Equilibria of Unknown Dynamics
Siddharth Chandak, Ilai Bistritz, Nicholas Bambos
Consider a decision-maker that can pick one out of actions to control an unknown system, for turns. The actions are interpreted as different configurations or policies. Hol…
Reinforcement Learning in Non-Markovian Environments
Siddharth Chandak, Pratik Shah, Vivek S Borkar +1
Motivated by the novel paradigm developed by Van Roy and coauthors for reinforcement learning in arbitrary non-Markovian environments, we propose a related formulation and explicit…
Learning to Speak on Behalf of a Group: Medium Access Control for Sending a Shared Message
Shaan ul Haque, Siddharth Chandak, Federico Chiariotti +2
The rapid development of Internet of Things (IoT) technologies has not only enabled new applications, but also presented new challenges for reliable communication with limited reso…
A Concentration Bound for LSPE()
Siddharth Chandak, Vivek S. Borkar, Harsh Dolhare
The popular LSPE() algorithm for policy evaluation is revisited to derive a concentration bound that gives high probability performance guarantees from some time on.
Concentration of Contractive Stochastic Approximation and Reinforcement Learning
Siddharth Chandak, Vivek S. Borkar, Parth Dodhia
Using a martingale concentration inequality, concentration bounds `from time on' are derived for stochastic approximation algorithms with contractive maps and both martingale…
Prospect-theoretic Q-learning
Vivek S. Borkar, Siddharth Chandak
We consider a prospect theoretic version of the classical Q-learning algorithm for discounted reward Markov decision processes, wherein the controller perceives a distorted and noi…
Hidden Markov Model-Based Encoding for Time-Correlated IoT Sources
Siddharth Chandak, Federico Chiariotti, Petar Popovski
As the use of Internet of Things (IoT) devices for monitoring purposes becomes ubiquitous, the efficiency of sensor communication is a major issue for the modern Internet. Channel…