6 papers · 1 filter
Limitations of SGD for Multi-Index Models Beyond Statistical Queries
Daniel Barzilai, Ohad Shamir
Understanding the limitations of gradient methods, and stochastic gradient descent (SGD) in particular, is a central challenge in learning theory. To that end, a commonly used tool…
When Is Compositional Reasoning Learnable from Verifiable Rewards?
Daniel Barzilai, Yotam Wolf, Ronen Basri
The emergence of compositional reasoning in large language models through reinforcement learning with verifiable rewards (RLVR) has been a key driver of recent empirical successes.…
Beyond Benign Overfitting in Nadaraya-Watson Interpolators
Daniel Barzilai, Guy Kornowski, Ohad Shamir
In recent years, there has been much interest in understanding the generalization behavior of interpolating predictors, which overfit on noisy training data. Whereas standard analy…
Querying Kernel Methods Suffices for Reconstructing their Training Data
Daniel Barzilai, Yuval Margalit, Eitan Gronich +3
Over-parameterized models have raised concerns about their potential to memorize training data, even when achieving strong generalization. The privacy implications of such memoriza…
Controlling the Inductive Bias of Wide Neural Networks by Modifying the Kernel's Spectrum
Amnon Geifman, Daniel Barzilai, Ronen Basri +1
Wide neural networks are biased towards learning certain functions, influencing both the rate of convergence of gradient descent (GD) and the functions that are reachable with GD i…
Generalization in Kernel Regression Under Realistic Assumptions
Daniel Barzilai, Ohad Shamir
It is by now well-established that modern over-parameterized models seem to elude the bias-variance tradeoff and generalize well despite overfitting noise. Many recent works attemp…