260 citations · 1.8k across the 63 of their papers we have counts for
10 papers · 1 filter
New Convergence Aspects of Stochastic Gradient Algorithms
Lam M. Nguyen, Phuong Ha Nguyen, Peter Richtárik +3
The classical convergence analysis of SGD is carried out under the assumption that the norm of the stochastic gradient is uniformly bounded. While this might hold for some loss fun…
Inexact SARAH Algorithm for Stochastic Optimization
Lam M. Nguyen, Katya Scheinberg, Martin Takáč
We develop and analyze a variant of the SARAH algorithm, which does not require computation of the exact gradient. Thus this new method can be applied to general expectation minimi…
Efficient Distributed Hessian Free Algorithm for Large-scale Empirical Risk Minimization via Accumulating Sample Strategy
Majid Jahani, Xi He, Chenxin Ma +4
In this paper, we propose a Distributed Accumulated Newton Conjugate gradiEnt (DANCE) method in which sample size is gradually increasing to quickly obtain a solution whose empiric…
Randomized sketch descent methods for non-separable linearly constrained optimization
Ion Necoara, Martin Takac
In this paper we consider large-scale smooth optimization problems with multiple linear coupled constraints. Due to the non-separability of the constraints, arbitrary random sketch…
On the Acceleration of L-BFGS with Second-Order Information and Stochastic Batches
Jie Liu, Yu Rong, Martin Takac +1
This paper proposes a framework of L-BFGS based on the (approximate) second-order information with stochastic batches, as a novel approach to the finite-sum minimization problems.…
Active Metric Learning for Supervised Classification
Krishnan Kumaran, Dimitri Papageorgiou, Yutong Chang +2
Clustering and classification critically rely on distance metrics that provide meaningful comparisons between data points. We present mixed-integer optimization approaches to find…
Entropy Penalized Semidefinite Programming
Mikhail Krechetov, Jakub Marecek, Yury Maximov +1
Low-rank methods for semidefinite programming (SDP) have gained a lot of interest recently, especially in machine learning applications. Their analysis often involves determinant-b…
Reinforcement Learning for Solving the Vehicle Routing Problem
Mohammadreza Nazari, Afshin Oroojlooy, Lawrence V. Snyder +1
We present an end-to-end framework for solving the Vehicle Routing Problem (VRP) using reinforcement learning. In this approach, we train a single model that finds near-optimal sol…
SGD and Hogwild! Convergence Without the Bounded Gradients Assumption
Lam M. Nguyen, Phuong Ha Nguyen, Marten van Dijk +3
Stochastic gradient descent (SGD) is the optimization algorithm of choice in many machine learning applications such as regularized empirical risk minimization and training deep ne…
Dual Free Adaptive Minibatch SDCA for Empirical Risk Minimization
Xi He, Rachael Tappenden, Martin Takac
In this paper we develop an adaptive dual free Stochastic Dual Coordinate Ascent (adfSDCA) algorithm for regularized empirical risk minimization problems. This is motivated by the…