362 citations · 2.5k across the 76 of their papers we have counts for
9 papers · 1 filter
Projection-Free Methods for Stochastic Simple Bilevel Optimization with Convex Lower-level Problem
Jincheng Cao, Ruichen Jiang, Nazanin Abolfazli +2
In this paper, we study a class of stochastic bilevel optimization problems, also known as stochastic simple bilevel optimization, where we minimize a smooth stochastic objective f…
Provable Multi-Task Representation Learning by Two-Layer ReLU Neural Networks
Liam Collins, Hamed Hassani, Mahdi Soltanolkotabi +2
An increasingly popular machine learning paradigm is to pretrain a neural network (NN) on many tasks offline, then adapt it to downstream tasks, often by re-training only the last…
Accelerated Quasi-Newton Proximal Extragradient: Faster Rate for Smooth Convex Optimization
Ruichen Jiang, Aryan Mokhtari
In this paper, we propose an accelerated quasi-Newton proximal extragradient (A-QPNE) method for solving unconstrained smooth convex optimization problems. With access only to the…
Limited-Memory Greedy Quasi-Newton Method with Non-asymptotic Superlinear Convergence Rate
Zhan Gao, Aryan Mokhtari, Alec Koppel
Non-asymptotic convergence analysis of quasi-Newton methods has gained attention with a landmark result establishing an explicit local superlinear rate of O. The…
An Inexact Conditional Gradient Method for Constrained Bilevel Optimization
Nazanin Abolfazli, Ruichen Jiang, Aryan Mokhtari +1
Bilevel optimization is an important class of optimization problems where one optimization problem is nested within another. While various methods have emerged to address unconstra…
Greedy Pruning with Group Lasso Provably Generalizes for Matrix Sensing
Nived Rajaraman, Devvrit, Aryan Mokhtari +1
Pruning schemes have been widely used in practice to reduce the complexity of trained models with a massive number of parameters. In fact, several practical studies have shown that…
InfoNCE Loss Provably Learns Cluster-Preserving Representations
Advait Parulekar, Liam Collins, Karthikeyan Shanmugam +2
The goal of contrasting learning is to learn a representation that preserves underlying clusters by keeping samples with similar content, e.g. the ``dogness'' of a dog, close to ea…
Online Learning Guided Curvature Approximation: A Quasi-Newton Method with Global Non-Asymptotic Superlinear Convergence
Ruichen Jiang, Qiujiang Jin, Aryan Mokhtari
Quasi-Newton algorithms are among the most popular iterative methods for solving unconstrained minimization problems, largely due to their favorable superlinear convergence propert…
Network Adaptive Federated Learning: Congestion and Lossy Compression
Parikshit Hegde, Gustavo de Veciana, Aryan Mokhtari
In order to achieve the dual goals of privacy and learning across distributed data, Federated Learning (FL) systems rely on frequent exchanges of large files (model updates) betwee…