362 citations · 2.5k across the 76 of their papers we have counts for
42 papers · 1 filter
Projection-Free Methods for Stochastic Simple Bilevel Optimization with Convex Lower-level Problem
Jincheng Cao, Ruichen Jiang, Nazanin Abolfazli +2
In this paper, we study a class of stochastic bilevel optimization problems, also known as stochastic simple bilevel optimization, where we minimize a smooth stochastic objective f…
Accelerated Quasi-Newton Proximal Extragradient: Faster Rate for Smooth Convex Optimization
Ruichen Jiang, Aryan Mokhtari
In this paper, we propose an accelerated quasi-Newton proximal extragradient (A-QPNE) method for solving unconstrained smooth convex optimization problems. With access only to the…
Limited-Memory Greedy Quasi-Newton Method with Non-asymptotic Superlinear Convergence Rate
Zhan Gao, Aryan Mokhtari, Alec Koppel
Non-asymptotic convergence analysis of quasi-Newton methods has gained attention with a landmark result establishing an explicit local superlinear rate of O. The…
An Inexact Conditional Gradient Method for Constrained Bilevel Optimization
Nazanin Abolfazli, Ruichen Jiang, Aryan Mokhtari +1
Bilevel optimization is an important class of optimization problems where one optimization problem is nested within another. While various methods have emerged to address unconstra…
Online Learning Guided Curvature Approximation: A Quasi-Newton Method with Global Non-Asymptotic Superlinear Convergence
Ruichen Jiang, Qiujiang Jin, Aryan Mokhtari
Quasi-Newton algorithms are among the most popular iterative methods for solving unconstrained minimization problems, largely due to their favorable superlinear convergence propert…
Conditional Gradient Methods
Gábor Braun, Alejandro Carderera, Cyrille W. Combettes +4
The purpose of this survey is to serve both as a gentle introduction and a coherent overview of state-of-the-art Frank--Wolfe algorithms, also called conditional gradient algorithm…
A Conditional Gradient-based Method for Simple Bilevel Optimization with Convex Lower-level Problem
Ruichen Jiang, Nazanin Abolfazli, Aryan Mokhtari +1
In this paper, we study a class of bilevel optimization problems, also known as simple bilevel optimization, where we minimize a smooth objective function over the optimal solution…
Statistical and Computational Complexities of BFGS Quasi-Newton Method for Generalized Linear Models
Qiujiang Jin, Tongzheng Ren, Nhat Ho +1
The gradient descent (GD) method has been used widely to solve parameter estimation in generalized linear models (GLMs), a generalization of linear models when the link function ca…
Sharpened Quasi-Newton Methods: Faster Superlinear Rate and Larger Local Convergence Neighborhood
Qiujiang Jin, Alec Koppel, Ketan Rajawat +1
Non-asymptotic analysis of quasi-Newton methods have gained traction recently. In particular, several works have established a non-asymptotic superlinear rate of $\mathcal{O}((1/\s…
Generalized Optimistic Methods for Convex-Concave Saddle Point Problems
Ruichen Jiang, Aryan Mokhtari
The optimistic gradient method has seen increasing popularity for solving convex-concave saddle point problems. To analyze its iteration complexity, a recent work [arXiv:1906.01115…
Minimax Optimization: The Case of Convex-Submodular
Arman Adibi, Aryan Mokhtari, Hamed Hassani
Minimax optimization has been central in addressing various applications in machine learning, game theory, and control theory. Prior literature has thus far mainly focused on study…
Exploiting Local Convergence of Quasi-Newton Methods Globally: Adaptive Sample Size Approach
Qiujiang Jin, Aryan Mokhtari
In this paper, we study the application of quasi-Newton methods for solving empirical risk minimization (ERM) problems defined over a large dataset. Traditional deterministic and s…
Non-asymptotic Superlinear Convergence of Standard Quasi-Newton Methods
Qiujiang Jin, Aryan Mokhtari
In this paper, we study and prove the non-asymptotic superlinear convergence rate of the Broyden class of quasi-Newton algorithms which includes the Davidon--Fletcher--Powell (DFP)…
A Decentralized Proximal Point-type Method for Saddle Point Problems
Weijie Liu, Aryan Mokhtari, Asuman Ozdaglar +3
In this paper, we focus on solving a class of constrained non-convex non-concave saddle point problems in a decentralized manner by a group of nodes in a network. Specifically, we…
One Sample Stochastic Frank-Wolfe
Mingrui Zhang, Zebang Shen, Aryan Mokhtari +2
One of the beauties of the projected gradient descent method lies in its rather simple mechanism and yet stable behavior with inexact, stochastic gradients, which has led to its wi…
DAve-QN: A Distributed Averaged Quasi-Newton Method with Local Superlinear Convergence Rate
Saeed Soori, Konstantin Mischenko, Aryan Mokhtari +2
In this paper, we consider distributed algorithms for solving the empirical risk minimization problem under the master/worker communication model. We develop a distributed asynchro…
Convergence Rate of for Optimistic Gradient and Extra-gradient Methods in Smooth Convex-Concave Saddle Point Problems
Aryan Mokhtari, Asuman Ozdaglar, Sarath Pattathil
We study the iteration complexity of the optimistic gradient descent-ascent (OGDA) method and the extra-gradient (EG) method for finding a saddle point of a convex-concave unconstr…
Stochastic Conditional Gradient++
Hamed Hassani, Amin Karbasi, Aryan Mokhtari +1
In this paper, we consider the general non-oblivious stochastic optimization where the underlying stochasticity may change during the optimization procedure and depends on the poin…
A Unified Analysis of Extra-gradient and Optimistic Gradient Methods for Saddle Point Problems: Proximal Point Approach
Aryan Mokhtari, Asuman Ozdaglar, Sarath Pattathil
In this paper we consider solving saddle point problems using two variants of Gradient Descent-Ascent algorithms, Extra-gradient (EG) and Optimistic Gradient Descent Ascent (OGDA)…
Achieving Acceleration in Distributed Optimization via Direct Discretization of the Heavy-Ball ODE
Jingzhao Zhang, César A. Uribe, Aryan Mokhtari +1
We develop a distributed algorithm for convex Empirical Risk Minimization, the problem of minimizing large but finite sum of convex functions over networks. The proposed algorithm…
A Primal-Dual Quasi-Newton Method for Exact Consensus Optimization
Mark Eisen, Aryan Mokhtari, Alejandro Ribeiro
We introduce the primal-dual quasi-Newton (PD-QN) method as an approximated second order method for solving decentralized optimization problems. The PD-QN method performs quasi-New…
Direct Runge-Kutta Discretization Achieves Acceleration
Jingzhao Zhang, Aryan Mokhtari, Suvrit Sra +1
We study gradient-based optimization methods obtained by directly discretizing a second-order ordinary differential equation (ODE) related to the continuous limit of Nesterov's acc…
Stochastic Conditional Gradient Methods: From Convex Minimization to Submodular Maximization
Aryan Mokhtari, Hamed Hassani, Amin Karbasi
This paper considers stochastic optimization problems for a large class of objective functions, including convex and continuous submodular. Stochastic proximal gradient methods hav…
Decentralized Submodular Maximization: Bridging Discrete and Continuous Settings
Aryan Mokhtari, Hamed Hassani, Amin Karbasi
In this paper, we showcase the interplay between discrete and continuous optimization in network-structured settings. We propose the first fully decentralized optimization method f…