99 citations · 510 across the 39 of their papers we have counts for
20 papers · 1 filter
Finding Optimal Policy for Queueing Models: New Parameterization
Trang H. Tran, Lam M. Nguyen, Katya Scheinberg
Queueing systems appear in many important real-life applications including communication networks, transportation and manufacturing systems. Reinforcement learning (RL) framework i…
StepDIRECT -- A Derivative-Free Optimization Method for Stepwise Functions
Dzung T. Phan, Hongsheng Liu, Lam M. Nguyen
In this paper, we propose the StepDIRECT algorithm for derivative-free optimization (DFO), in which the black-box objective function has a stepwise landscape. Our framework is base…
On Unbalanced Optimal Transport: Gradient Methods, Sparsity and Approximation Error
Quang Minh Nguyen, Hoang H. Nguyen, Yi Zhou +1
We study the Unbalanced Optimal Transport (UOT) between two measures of possibly different masses with at most components, where the marginal constraints of standard Optimal Tr…
Nesterov Accelerated Shuffling Gradient Method for Convex Optimization
Trang H. Tran, Katya Scheinberg, Lam M. Nguyen
In this paper, we propose Nesterov Accelerated Shuffling Gradient (NASG), a new algorithm for the convex finite-sum minimization problems. Our method integrates the traditional Nes…
SMG: A Shuffling Gradient-Based Method with Momentum
Trang H. Tran, Lam M. Nguyen, Quoc Tran-Dinh
We combine two advanced ideas widely used in optimization for machine learning: shuffling strategy and momentum technique to develop a novel shuffling gradient-based method with mo…
An Optimal Hybrid Variance-Reduced Algorithm for Stochastic Composite Nonconvex Optimization
Deyi Liu, Lam M. Nguyen, Quoc Tran-Dinh
In this note we propose a new variant of the hybrid variance-reduced proximal gradient method in [7] to solve a common stochastic composite nonconvex optimization problem under sta…
Hybrid Variance-Reduced SGD Algorithms For Nonconvex-Concave Minimax Problems
Quoc Tran-Dinh, Deyi Liu, Lam M. Nguyen
We develop a novel and single-loop variance-reduced algorithm to solve a class of stochastic nonconvex-convex minimax problems involving a nonconvex-linear objective function, whic…
Finite-Time Analysis of Stochastic Gradient Descent under Markov Randomness
Thinh T. Doan, Lam M. Nguyen, Nhan H. Pham +1
Motivated by broad applications in reinforcement learning and machine learning, this paper considers the popular stochastic gradient descent (SGD) when the gradients of the underly…
A Unified Convergence Analysis for Shuffling-Type Gradient Methods
Lam M. Nguyen, Quoc Tran-Dinh, Dzung T. Phan +2
In this paper, we propose a unified convergence analysis for a class of generic shuffling-type gradient methods for solving finite-sum optimization problems. Our analysis works wit…
Stochastic Gauss-Newton Algorithms for Nonconvex Compositional Optimization
Quoc Tran-Dinh, Nhan H. Pham, Lam M. Nguyen
We develop two new stochastic Gauss-Newton algorithms for solving a class of non-convex stochastic compositional optimization problems frequently arising in practice. We consider b…
Convergence Rates of Accelerated Markov Gradient Descent with Applications in Reinforcement Learning
Thinh T. Doan, Lam M. Nguyen, Nhan H. Pham +1
Motivated by broad applications in machine learning, we study the popular accelerated stochastic gradient descent (ASGD) algorithm for solving (possibly nonconvex) optimization pro…
A Hybrid Stochastic Optimization Framework for Stochastic Composite Nonconvex Optimization
Quoc Tran-Dinh, Nhan H. Pham, Dzung T. Phan +1
We introduce a new approach to develop stochastic optimization algorithms for a class of stochastic composite and possibly nonconvex optimization problems. The main idea is to comb…
Hybrid Stochastic Gradient Descent Algorithms for Stochastic Nonconvex Optimization
Quoc Tran-Dinh, Nhan H. Pham, Dzung T. Phan +1
We introduce a hybrid stochastic estimator to design stochastic gradient algorithms for solving stochastic optimization problems. Such a hybrid estimator is a convex combination of…
ProxSARAH: An Efficient Algorithmic Framework for Stochastic Composite Nonconvex Optimization
Nhan H. Pham, Lam M. Nguyen, Dzung T. Phan +1
We propose a new stochastic first-order algorithmic framework to solve stochastic composite nonconvex optimization problems that covers both finite-sum and expectation settings. Ou…
Finite-Sum Smooth Optimization with SARAH
Lam M. Nguyen, Marten van Dijk, Dzung T. Phan +3
The total complexity (measured as the total number of gradient computations) of a stochastic first-order optimization algorithm that finds a first-order stationary point of a finit…
New Convergence Aspects of Stochastic Gradient Algorithms
Lam M. Nguyen, Phuong Ha Nguyen, Peter Richtárik +3
The classical convergence analysis of SGD is carried out under the assumption that the norm of the stochastic gradient is uniformly bounded. While this might hold for some loss fun…
Inexact SARAH Algorithm for Stochastic Optimization
Lam M. Nguyen, Katya Scheinberg, Martin Takáč
We develop and analyze a variant of the SARAH algorithm, which does not require computation of the exact gradient. Thus this new method can be applied to general expectation minimi…
Tight Dimension Independent Lower Bound on the Expected Convergence Rate for Diminishing Step Sizes in SGD
Phuong Ha Nguyen, Lam M. Nguyen, Marten van Dijk
We study the convergence of Stochastic Gradient Descent (SGD) for strongly convex objective functions. We prove for all a lower bound on the expected convergence rate after the…
Characterization of Convex Objective Functions and Optimal Expected Convergence Rates for SGD
Marten van Dijk, Lam M. Nguyen, Phuong Ha Nguyen +1
We study Stochastic Gradient Descent (SGD) with diminishing step sizes for convex objective functions. We introduce a definitional framework and theory that defines and characteriz…
SGD and Hogwild! Convergence Without the Bounded Gradients Assumption
Lam M. Nguyen, Phuong Ha Nguyen, Marten van Dijk +3
Stochastic gradient descent (SGD) is the optimization algorithm of choice in many machine learning applications such as regularized empirical risk minimization and training deep ne…