6 papers
Limitations of SGD for Multi-Index Models Beyond Statistical Queries
Daniel Barzilai, Ohad Shamir
Understanding the limitations of gradient methods, and stochastic gradient descent (SGD) in particular, is a central challenge in learning theory. To that end, a commonly used tool…
When Models Don't Collapse: On the Consistency of Iterative MLE
Daniel Barzilai, Ohad Shamir
The widespread use of generative models has created a feedback loop, in which each generation of models is trained on data partially produced by its predecessors. This process has…
When Is Compositional Reasoning Learnable from Verifiable Rewards?
Daniel Barzilai, Yotam Wolf, Ronen Basri
The emergence of compositional reasoning in large language models through reinforcement learning with verifiable rewards (RLVR) has been a key driver of recent empirical successes.…
Beyond Benign Overfitting in Nadaraya-Watson Interpolators
Daniel Barzilai, Guy Kornowski, Ohad Shamir
In recent years, there has been much interest in understanding the generalization behavior of interpolating predictors, which overfit on noisy training data. Whereas standard analy…
Simple Relative Deviation Bounds for Covariance and Gram Matrices
Daniel Barzilai, Ohad Shamir
We provide non-asymptotic, relative deviation bounds for the eigenvalues of empirical covariance and Gram matrices in general settings. Unlike typical uniform bounds, which may fai…
Querying Kernel Methods Suffices for Reconstructing their Training Data
Daniel Barzilai, Yuval Margalit, Eitan Gronich +3
Over-parameterized models have raised concerns about their potential to memorize training data, even when achieving strong generalization. The privacy implications of such memoriza…