1.3k citations · 3.8k across the 222 of their papers we have counts for
6 papers · 1 filter
Subquadratic Overparameterization for Shallow Neural Networks
Chaehwan Song, Ali Ramezani-Kebrya, Thomas Pethick +2
Overparameterization refers to the important phenomenon where the width of a neural network is chosen such that learning algorithms can provably attain zero loss in nonconvex train…
STORM+: Fully Adaptive SGD with Momentum for Nonconvex Optimization
Kfir Y. Levy, Ali Kavis, Volkan Cevher
In this work we investigate stochastic non-convex optimization problems where the objective is an expectation over smooth loss functions, and the goal is to find an approximate sta…
A first-order primal-dual method with adaptivity to local smoothness
Maria-Luiza Vladarean, Yura Malitsky, Volkan Cevher
We consider the problem of finding a saddle point for the convex-concave objective , where is a convex function with locally…
On the Double Descent of Random Features Models Trained with SGD
Fanghui Liu, Johan A. K. Suykens, Volkan Cevher
We study generalization properties of random features (RF) regression in high dimensions optimized by stochastic gradient descent (SGD) in under-/over-parameterized regime. In this…
Self-Supervised Neural Architecture Search for Imbalanced Datasets
Aleksandr Timofeev, Grigorios G. Chrysos, Volkan Cevher
Neural Architecture Search (NAS) provides state-of-the-art results when trained on well-curated datasets with annotated labels. However, annotating data or even having balanced num…
On the Generalization of Stochastic Gradient Descent with Momentum
Ali Ramezani-Kebrya, Ashish Khisti, Ben Liang
While momentum-based methods, in conjunction with stochastic gradient descent (SGD), are widely used when training machine learning models, there is little theoretical understandin…