28 citations · 78 across the 16 of their papers we have counts for
16 papers
Non-Asymptotic Convergence of Stochastic Iterative Algorithms: A Lyapunov Framework
Zaiwei Chen, Siva Theja Maguluri
We survey Lyapunov-based techniques for the finite-time analysis of stochastic iterative algorithms, also known as stochastic approximation (SA) algorithms, for solving fixed-point…
A Non-Asymptotic Theory of Seminorm Lyapunov Stability: From Deterministic to Stochastic Iterative Algorithms
Zaiwei Chen, Sheng Zhang, Zhe Zhang +2
We study the problem of solving fixed-point equations for seminorm-contractive operators and establish foundational results on the non-asymptotic behavior of iterative algorithms i…
A Finite-Sample Analysis of Payoff-Based Independent Learning in Zero-Sum Stochastic Games
Zaiwei Chen, Kaiqing Zhang, Eric Mazumdar +2
We study two-player zero-sum stochastic games, and propose a form of independent learning dynamics called Doubly Smoothed Best-Response dynamics, which integrates a discrete and do…
Concentration of Contractive Stochastic Approximation: Additive and Multiplicative Noise
Zaiwei Chen, Siva Theja Maguluri, Martin Zubeldia
In this paper, we establish maximal concentration bounds for the iterates generated by a stochastic approximation (SA) algorithm under a contractive operator with respect to some a…
Convergence Rates for Localized Actor-Critic in Networked Markov Potential Games
Zhaoyi Zhou, Zaiwei Chen, Yiheng Lin +1
We introduce a class of networked Markov potential games in which agents are associated with nodes in a network. Each agent has its own local potential function, and the reward of…
Global Convergence of Localized Policy Iteration in Networked Multi-Agent Reinforcement Learning
Yizhou Zhang, Guannan Qu, Pan Xu +3
We study a multi-agent reinforcement learning (MARL) problem where the agents interact over a given network. The goal of the agents is to cooperatively maximize the average of thei…
An Approximate Policy Iteration Viewpoint of Actor-Critic Algorithms
Zaiwei Chen, Siva Theja Maguluri
In this work, we consider policy-based methods for solving the reinforcement learning problem, and establish the sample complexity guarantees. A policy-based algorithm typically co…
Target Network and Truncation Overcome The Deadly Triad in -Learning
Zaiwei Chen, John Paul Clarke, Siva Theja Maguluri
-learning with function approximation is one of the most empirically successful while theoretically mysterious reinforcement learning (RL) algorithms, and was identified in Sutt…
Stationary Behavior of Constant Stepsize SGD Type Algorithms: An Asymptotic Characterization
Zaiwei Chen, Shancong Mou, Siva Theja Maguluri
Stochastic approximation (SA) and stochastic gradient descent (SGD) algorithms are work-horses for modern machine learning algorithms. Their constant stepsize variants are preferre…
Finite-Sample Analysis of Off-Policy TD-Learning via Generalized Bellman Operators
Zaiwei Chen, Siva Theja Maguluri, Sanjay Shakkottai +1
In temporal difference (TD) learning, off-policy sampling is known to be more practical than on-policy sampling, and by decoupling learning from data collection, it enables data re…
Finite-Sample Analysis of Off-Policy Natural Actor-Critic with Linear Function Approximation
Zaiwei Chen, Sajad Khodadadian, Siva Theja Maguluri
In this paper, we develop a novel variant of off-policy natural actor-critic algorithm with linear function approximation and we establish a sample complexity of $\mathcal{O}(ε^{-3…
Nested Vehicle Routing Problem: Optimizing Drone-Truck Surveillance Operations
Fanruiqi Zeng, Zaiwei Chen, John-Paul Clarke +1
Unmanned aerial vehicles or drones are becoming increasingly popular due to their low cost and high mobility. In this paper we address the routing and coordination of a drone-truck…
Finite-Sample Analysis of Off-Policy Natural Actor-Critic Algorithm
Sajad Khodadadian, Zaiwei Chen, Siva Theja Maguluri
In this paper, we provide finite-sample convergence guarantees for an off-policy variant of the natural actor-critic (NAC) algorithm based on Importance Sampling. In particular, we…
A Lyapunov Theory for Finite-Sample Guarantees of Asynchronous Q-Learning and TD-Learning Variants
Zaiwei Chen, Siva Theja Maguluri, Sanjay Shakkottai +1
This paper develops an unified framework to study finite-sample convergence guarantees of a large class of value-based asynchronous reinforcement learning (RL) algorithms. We do th…
Finite-Sample Analysis of Stochastic Approximation Using Smooth Convex Envelopes
Zaiwei Chen, Siva Theja Maguluri, Sanjay Shakkottai +1
Stochastic Approximation (SA) is a popular approach for solving fixed-point equations where the information is corrupted by noise. In this paper, we consider an SA involving a cont…
Finite-Sample Analysis of Nonlinear Stochastic Approximation with Applications in Reinforcement Learning
Zaiwei Chen, Sheng Zhang, Thinh T. Doan +2
Motivated by applications in reinforcement learning (RL), we study a nonlinear stochastic approximation (SA) algorithm under Markovian noise, and establish its finite-sample conver…