50 citations · 194 across the 41 of their papers we have counts for
29 papers · 1 filter
Accelerated Multi-Time-Scale Stochastic Approximation: Optimal Complexity and Applications in Reinforcement Learning and Multi-Agent Games
Sihan Zeng, Thinh T. Doan
Multi-time-scale stochastic approximation is an iterative algorithm for finding the fixed point of a set of coupled operators given their noisy samples. It has been observed th…
Resilient Two-Time-Scale Local Stochastic Gradient Descent for Byzantine Federated Learning
Amit Dutta, Thinh T. Doan
We study local stochastic gradient descent methods for solving federated optimization over a network of agents communicating indirectly through a centralized coordinator. We are in…
Natural Policy Gradient and Actor Critic Methods for Constrained Multi-Task Reinforcement Learning
Sihan Zeng, Thinh T. Doan, Justin Romberg
Multi-task reinforcement learning (RL) aims to find a single policy that effectively solves multiple tasks at the same time. This paper presents a constrained formulation for multi…
Fast Two-Time-Scale Stochastic Gradient Method with Applications in Reinforcement Learning
Sihan Zeng, Thinh T. Doan
Two-time-scale optimization is a framework introduced in Zeng et al. (2024) that abstracts a range of policy evaluation and policy optimization problems in reinforcement learning (…
Fast Nonlinear Two-Time-Scale Stochastic Approximation: Achieving Finite-Sample Complexity
Thinh T. Doan
This paper proposes to develop a new variant of the two-time-scale stochastic approximation to find the roots of two coupled nonlinear operators, assuming only noisy samples of the…
Resilient Federated Learning under Byzantine Attack in Distributed Nonconvex Optimization with 2-f Redundancy
Amit Dutta, Thinh T. Doan, Jeffrey H. Reed
We study the problem of Byzantine fault tolerance in a distributed optimization setting, where there is a group of agents communicating with a trusted centralized coordinator.…
Regularized Gradient Descent Ascent for Two-Player Zero-Sum Markov Games
Sihan Zeng, Thinh T. Doan, Justin Romberg
We study the problem of finding the Nash equilibrium in a two-player zero-sum Markov game. Due to its formulation as a minimax optimization program, a natural approach to solve the…
Convergence Rates of Two-Time-Scale Gradient Descent-Ascent Dynamics for Solving Nonconvex Min-Max Problems
Thinh T. Doan
There are much recent interests in solving noncovnex min-max optimization problems due to its broad applications in many areas including machine learning, networked resource alloca…
Convergence Rates of Decentralized Gradient Methods over Cluster Networks
Amit Dutta, Nila Masrourisaadat, Thinh T. Doan
We present an analysis for the performance of decentralized consensus-based gradient (DCG) methods for solving optimization problems over a cluster network of nodes. This type of n…
Finite-Time Complexity of Online Primal-Dual Natural Actor-Critic Algorithm for Constrained Markov Decision Processes
Sihan Zeng, Thinh T. Doan, Justin Romberg
We consider a discounted cost constrained Markov decision process (CMDP) policy optimization problem, in which an agent seeks to maximize a discounted cumulative reward subject to…
A Two-Time-Scale Stochastic Optimization Framework with Applications in Control and Reinforcement Learning
Sihan Zeng, Thinh T. Doan, Justin Romberg
We study a new two-time-scale stochastic gradient method for solving optimization problems, where the gradients are computed with the aid of an auxiliary variable under samples gen…
Distributed Dual Subgradient Methods with Averaging and Applications to Grid Optimization
Subhonmesh Bose, Hoa Dinh Nguyen, Haitian Liu +3
We study finite-time performance of a recently proposed distributed dual subgradient (DDSG) method for convex constrained multi-agent optimization problems. The algorithm enjoys pe…
Finite-Time Convergence Rates of Nonlinear Two-Time-Scale Stochastic Approximation under Markovian Noise
Thinh T. Doan
We study the so-called two-time-scale stochastic approximation, a simulation-based approach for finding the roots of two coupled nonlinear operators. Our focus is to characterize i…
Convergence Rates of Distributed Consensus over Cluster Networks: A Two-Time-Scale Approach
Amit Dutta, Almuatazbellah M. Boker, Thinh T. Doan
We study the popular distributed consensus method over networks composed of a number of densely connected clusters with a sparse connection between them. In these cluster networks,…
Nonlinear Two-Time-Scale Stochastic Approximation: Convergence and Finite-Time Performance
Thinh T. Doan
Two-time-scale stochastic approximation, a generalized version of the popular stochastic approximation, has found broad applications in many areas including stochastic control, opt…
Finite-Time Analysis of Stochastic Gradient Descent under Markov Randomness
Thinh T. Doan, Lam M. Nguyen, Nhan H. Pham +1
Motivated by broad applications in reinforcement learning and machine learning, this paper considers the popular stochastic gradient descent (SGD) when the gradients of the underly…
Convergence Rates of Accelerated Markov Gradient Descent with Applications in Reinforcement Learning
Thinh T. Doan, Lam M. Nguyen, Nhan H. Pham +1
Motivated by broad applications in machine learning, we study the popular accelerated stochastic gradient descent (ASGD) algorithm for solving (possibly nonconvex) optimization pro…
Finite-Time Performance of Distributed Two-Time-Scale Stochastic Approximation
Thinh T. Doan, Justin Romberg
Two-time-scale stochastic approximation is a popular iterative method for finding the solution of a system of two equations. Such methods have found broad applications in many area…
Finite-Time Performance of Distributed Temporal Difference Learning with Linear Function Approximation
Thinh T. Doan, Siva Theja Maguluri, Justin Romberg
We study the policy evaluation problem in multi-agent reinforcement learning, modeled by a Markov decision process. In this problem, the agents operate in a common environment unde…
Finite-Sample Analysis of Nonlinear Stochastic Approximation with Applications in Reinforcement Learning
Zaiwei Chen, Sheng Zhang, Thinh T. Doan +2
Motivated by applications in reinforcement learning (RL), we study a nonlinear stochastic approximation (SA) algorithm under Markovian noise, and establish its finite-sample conver…
Finite-Time Analysis of Distributed TD(0) with Linear Function Approximation for Multi-Agent Reinforcement Learning
Thinh T. Doan, Siva Theja Maguluri, Justin Romberg
We study the policy evaluation problem in multi-agent reinforcement learning. In this problem, a group of agents works cooperatively to evaluate the value function for the global d…
Distributed Stochastic Approximation for Solving Network Optimization Problems Under Random Quantization
Thinh T. Doan, Siva Theja Maguluri, Justin Romberg
We study distributed optimization problems over a network when the communication between the nodes is constrained, and so information that is exchanged between the nodes must be qu…
Fast Convergence Rates of Distributed Subgradient Methods with Adaptive Quantization
Thinh T. Doan, Siva Theja Maguluri, Justin Romberg
We study distributed optimization problems over a network when the communication between the nodes is constrained, and so information that is exchanged between the nodes must be qu…
Convergence of the Iterates in Mirror Descent Methods
Thinh T. Doan, Subhonmesh Bose, D. Hoa Nguyen +1
We consider centralized and distributed mirror descent algorithms over a finite-dimensional Hilbert space, and prove that the problem variables converge to an optimizer of a possib…