24 citations · 75 across the 13 of their papers we have counts for
14 papers
An Accelerated Primal Dual Algorithm with Backtracking for Decentralized Constrained Optimization
Qiushui Xu, Necdet Serhat Aybat, Mert Gürbüzbalaban
We propose a distributed accelerated primal-dual method with backtracking (D-APDB) for cooperative multi-agent constrained consensus optimization problems over an undirected networ…
High-probability complexity guarantees for nonconvex minimax problems
Yassine Laguel, Yasa Syed, Necdet Serhat Aybat +1
Stochastic smooth nonconvex minimax problems are prevalent in machine learning, e.g., GAN training, fair classification, and distributionally robust learning. Stochastic gradient d…
A Stochastic GDA Method With Backtracking For Solving Nonconvex Concave Minimax Problems
Necdet Serhat Aybat, Qiushui Xu, Xuan Zhang +1
We propose a stochastic GDA (gradient descent ascent) method with backtracking (SGDA-B) to solve nonconvex-concave (NCC) minimax problems of the form: $\min_{\mathbf{x}} \max_y \su…
Privacy of SGD under Gaussian or Heavy-Tailed Noise: Guarantees without Gradient Clipping
Umut Şimşekli, Mert Gürbüzbalaban, Sinan Yıldırım +1
The injection of heavy-tailed noise into the iterates of stochastic gradient descent (SGD) has garnered growing interest in recent years due to its theoretical and empirical benefi…
Distributionally Robust Learning with Weakly Convex Losses: Convergence Rates and Finite-Sample Guarantees
Landi Zhu, Mert Gürbüzbalaban, Andrzej Ruszczyński
We consider a distributionally robust stochastic optimization problem and formulate it as a stochastic two-level composition optimization problem with the use of the mean--semidevi…
Heavy-Tail Phenomenon in Decentralized SGD
Mert Gurbuzbalaban, Yuanhan Hu, Umut Simsekli +2
Recent theoretical studies have shown that heavy-tails can emerge in stochastic optimization due to `multiplicative noise', even under surprisingly simple settings, such as linear…
Fractal Structure and Generalization Properties of Stochastic Optimization Algorithms
Alexander Camuto, George Deligiannidis, Murat A. Erdogdu +3
Understanding generalization in deep learning has been one of the major challenges in statistical learning theory over the last decade. While recent work has illustrated that the d…
Convergence Rates of Stochastic Gradient Descent under Infinite Noise Variance
Hongjian Wang, Mert Gürbüzbalaban, Lingjiong Zhu +2
Recent studies have provided both empirical and theoretical evidence illustrating that heavy tails can emerge in stochastic gradient descent (SGD) in various scenarios. Such heavy…
Non-Convex Optimization via Non-Reversible Stochastic Gradient Langevin Dynamics
Yuanhan Hu, Xiaoyu Wang, Xuefeng Gao +2
Stochastic Gradient Langevin Dynamics (SGLD) is a powerful algorithm for optimizing a non-convex objective, where a controlled and properly scaled Gaussian noise is added to the st…
Robust Distributed Accelerated Stochastic Gradient Methods for Multi-Agent Networks
Alireza Fallah, Mert Gurbuzbalaban, Asuman Ozdaglar +2
We study distributed stochastic gradient (D-SG) method and its accelerated variant (D-ASG) for solving decentralized strongly convex stochastic optimization problems where the obje…
DAve-QN: A Distributed Averaged Quasi-Newton Method with Local Superlinear Convergence Rate
Saeed Soori, Konstantin Mischenko, Aryan Mokhtari +2
In this paper, we consider distributed algorithms for solving the empirical risk minimization problem under the master/worker communication model. We develop a distributed asynchro…
Randomness and Permutations in Coordinate Descent Methods
Mert Gurbuzbalaban, Asuman Ozdaglar, Nuri Denizcan Vanli +1
We consider coordinate descent (CD) methods with exact line search on convex quadratic problems. Our main focus is to study the performance of the CD method that use random permuta…
A Stronger Convergence Result on the Proximal Incremental Aggregated Gradient Method
Nuri Denizcan Vanli, Mert Gurbuzbalaban, Asu Ozdaglar
We study the convergence rate of the proximal incremental aggregated gradient (PIAG) method for minimizing the sum of a large number of smooth component functions (where the sum is…
Global Convergence Rate of Proximal Incremental Aggregated Gradient Methods
Nuri Denizcan Vanli, Mert Gurbuzbalaban, Asu Ozdaglar
We focus on the problem of minimizing the sum of smooth component functions (where the sum is strongly convex) and a non-smooth convex function, which arises in regularized empiric…