3 citations · 3 across the 9 of their papers we have counts for
11 papers
Benign Overfitting for General Norms and Distributions
Daniel Barzilai, Ohad Shamir
Understanding why predictors can generalize despite interpolating noisy training data is a central puzzle in machine learning. Most work on such "benign overfitting" studies minimu…
Gradient Descent's Last Iterate is Often (slightly) Suboptimal
Guy Kornowski, Ohad Shamir
We consider the well-studied setting of minimizing a convex Lipschitz function using either gradient descent (GD) or its stochastic variant (SGD), and examine the last iterate conv…
Limitations of SGD for Multi-Index Models Beyond Statistical Queries
Daniel Barzilai, Ohad Shamir
Understanding the limitations of gradient methods, and stochastic gradient descent (SGD) in particular, is a central challenge in learning theory. To that end, a commonly used tool…
When Models Don't Collapse: On the Consistency of Iterative MLE
Daniel Barzilai, Ohad Shamir
The widespread use of generative models has created a feedback loop, in which each generation of models is trained on data partially produced by its predecessors. This process has…
Are Convex Optimization Curves Convex?
Guy Barzilai, Ohad Shamir, Moslem Zamani
In this paper, we study when we might expect the optimization curve induced by gradient descent to be \emph{convex} -- precluding, for example, an initial plateau followed by a sha…
Beyond Benign Overfitting in Nadaraya-Watson Interpolators
Daniel Barzilai, Guy Kornowski, Ohad Shamir
In recent years, there has been much interest in understanding the generalization behavior of interpolating predictors, which overfit on noisy training data. Whereas standard analy…
The Oracle Complexity of Simplex-based Matrix Games
Guy Kornowski, Ohad Shamir
We study the problem of solving matrix games of the form , where is a matrix and is the pr…
Simple Relative Deviation Bounds for Covariance and Gram Matrices
Daniel Barzilai, Ohad Shamir
We provide non-asymptotic, relative deviation bounds for the eigenvalues of empirical covariance and Gram matrices in general settings. Unlike typical uniform bounds, which may fai…
On the Hardness of Meaningful Local Guarantees in Nonsmooth Nonconvex Optimization
Guy Kornowski, Swati Padmanabhan, Ohad Shamir
We study the oracle complexity of nonsmooth nonconvex optimization, with the algorithm assumed to have access only to local function information. It has been shown by Davis, Drusvy…
Open Problem: Anytime Convergence Rate of Gradient Descent
Guy Kornowski, Ohad Shamir
Recent results show that vanilla gradient descent can be accelerated for smooth convex objectives, merely by changing the stepsize sequence. We show that this can lead to surprisin…
Accelerated Zeroth-order Method for Non-Smooth Stochastic Convex Optimization Problem with Infinite Variance
Nikita Kornilov, Ohad Shamir, Aleksandr Lobanov +5
In this paper, we consider non-smooth stochastic convex optimization with two function evaluations per round under infinite noise variance. In the classical setting when noise has…