50 citations · 222 across the 56 of their papers we have counts for
60 papers
Transform Method for Stochastic Processing and Matching Networks
Sushil Mahavir Varma, Prakirt Jhunjhunwala, Daniela Hurtado-Lange +1
Modern service systems, ranging from cloud data centers and ride-hailing platforms to healthcare facilities, operate at massive scales where it is important to handle congestion. Q…
Non-Asymptotic Convergence of Stochastic Iterative Algorithms: A Lyapunov Framework
Zaiwei Chen, Siva Theja Maguluri
We survey Lyapunov-based techniques for the finite-time analysis of stochastic iterative algorithms, also known as stochastic approximation (SA) algorithms, for solving fixed-point…
Concentration of General Stochastic Approximation Under Heavy-Tailed Markovian Noise
Shubhada Agrawal, Siva Theja Maguluri, Martin Zubeldia
We establish maximal concentration bounds for the iterates generated by stochastic approximation algorithms with general step sizes, where the noise has a finite-state Markovian co…
Tail Bounds for Queues with Abandonment: Constant, Moderate, Large Deviations, and Efficient Concentration
Zedong Wang, Siva Theja Maguluri
We study a heavily overloaded single-server queue with abandonment and derive bounds on stationary tail probabilities of the queue length. As the abandonment rate ,…
Almost Sure Convergence of Stochastic Approximation: An Interplay of Noise and Step Size
Quang Dinh Thien Nguyen, Duc Anh Nguyen, Hoang Huy Nguyen +1
We study the almost sure convergence of the Stochastic Approximation algorithm to the fixed point of a nonlinear operator under a negative drift condition and a general n…
Constant-Stepsize Stochastic Approximation: Finite-Time Convergence, Gaussian Approximation, and Tail Bounds
Zedong Wang, Yuyang Wang, Ijay Narang +3
Constant-stepsize stochastic approximation (SA) is widely used in learning for computational efficiency, yet the distribution of the iterates is typically intractable. Classical as…
How Accurately Can a Gaussian Approximate Stochastic Approximation Iterates?
Shaan Ul Haque, Zedong Wang, Zixuan Zhang +1
Stochastic approximation (SA) is a method for finding the root of an operator perturbed by noise. The focus of this paper is studying the distribution of SA iterates in finite time…
Higher-Order Approximations of Sojourn Times in M/G/1 Queues via Stein's Method
Bihan Chatterjee, Siva Theja Maguluri, Debankur Mukherjee
We study the stationary sojourn time distribution in an M/G/1 queue operating under heavy traffic. It is known that the sojourn time converges to an exponential distribution in the…
Finite-Time Behavior of Erlang-C Model: Mixing Time, Mean Queue Length and Tail Bounds
Hoang Huy Nguyen, Sushil Mahavir Varma, Siva Theja Maguluri
Resource allocation problems in service systems like data centers and ride-hailing are usually studied using queueing models. Such systems are primarily studied in the steady-state…
Convergence Rate of the Join-the-Shortest-Queue System
Yuanzhe Ma, Siva Theja Maguluri
The Join-the-Shortest-Queue (JSQ) policy is among the most widely used load balancing algorithms and has been extensively studied. However, an exact characterization of the system…
A Non-Asymptotic Theory of Seminorm Lyapunov Stability: From Deterministic to Stochastic Iterative Algorithms
Zaiwei Chen, Sheng Zhang, Zhe Zhang +2
We study the problem of solving fixed-point equations for seminorm-contractive operators and establish foundational results on the non-asymptotic behavior of iterative algorithms i…
Stochastic Approximation with Unbounded Markovian Noise: A General-Purpose Theorem
Shaan Ul Haque, Siva Theja Maguluri
Motivated by engineering applications such as resource allocation in networks and inventory systems, we consider average-reward Reinforcement Learning with unbounded state space an…
Markov Chain Variance Estimation: A Stochastic Approximation Approach
Shubhada Agrawal, Prashanth L. A., Siva Theja Maguluri
We consider the problem of estimating the asymptotic variance of a function defined on a Markov chain, an important step for statistical inference of the stationary mean. We design…
Performance of NPG in Countable State-Space Average-Cost RL
Yashaswini Murthy, Isaac Grosof, Siva Theja Maguluri +1
We consider policy optimization methods in reinforcement learning settings where the state space is arbitrarily large, or even countably infinite. The motivation arises from contro…
Convergence of Natural Policy Gradient for a Family of Infinite-State Queueing MDPs
Isaac Grosof, Siva Theja Maguluri, R. Srikant
A wide variety of queueing systems can be naturally modeled as infinite-state Markov Decision Processes (MDPs). In the reinforcement learning (RL) context, a variety of algorithms…
Tight Finite Time Bounds of Two-Time-Scale Linear Stochastic Approximation with Markovian Noise
Shaan Ul Haque, Sajad Khodadadian, Siva Theja Maguluri
Stochastic approximation (SA) is an iterative algorithm for finding the fixed point of an operator using noisy samples and widely used in optimization and Reinforcement Learning (R…
Exponential Tail Bounds on Queues: A Confluence of Non-Asymptotic Heavy Traffic and Large Deviations
Prakirt Raj Jhunjhunwala, Daniela Hurtado-Lange, Siva Theja Maguluri
In general, obtaining the exact steady-state distribution of queue lengths is not feasible. Therefore, we establish bounds for the tail probabilities of queue lengths. Specifically…
Electric Vehicle Fleet and Charging Infrastructure Planning
Sushil Mahavir Varma, Francisco Castro, Siva Theja Maguluri
We study electric vehicle (EV) fleet and charging infrastructure planning in a spatial setting. With customer requests arriving continuously at rate throughout the day, we dete…
Stochastic Approximation for Nonlinear Discrete Stochastic Control: Finite-Sample Bounds
Hoang Huy Nguyen, Siva Theja Maguluri
We consider a nonlinear discrete stochastic control system, and our goal is to design a feedback control policy in order to lead the system to a prespecified state. We adopt a stoc…
Concentration of Contractive Stochastic Approximation: Additive and Multiplicative Noise
Zaiwei Chen, Siva Theja Maguluri, Martin Zubeldia
In this paper, we establish maximal concentration bounds for the iterates generated by a stochastic approximation (SA) algorithm under a contractive operator with respect to some a…
Backward Curriculum Reinforcement Learning
KyungMin Ko
Current reinforcement learning algorithms train an agent using forward-generated trajectories, which provide little guidance so that the agent can explore as much as possible. Whil…
Join-the-Shortest Queue with Abandonment: Critically Loaded and Heavily Overloaded Regimes
Prakirt Raj Jhunjhunwala, Martin Zubeldia, Siva Theja Maguluri
We consider a load balancing system comprised of a fixed number of single server queues, operating under the well-known Join-the-Shortest Queue policy, and where jobs/customers are…
Matching Queues with Abandonments in Quantum Switches: Stability and Throughput Analysis
Martin Zubeldia, Prakirt R. Jhunjhunwala, Siva Theja Maguluri
Inspired by quantum switches, we consider a discrete-time multi-way matching system with two classes of arrivals: requests for entangled pair of qubits between two nodes, and qubit…
Power-of- Choices Load Balancing in the Sub-Halfin Whitt Regime
Sushil Mahavir Varma, Francisco Castro, Siva Theja Maguluri
We consider the load balancing system under Poisson arrivals, exponential services, and homogeneous servers. Upon arrival, a job is to be routed to one of the servers, where it is…