5 papers
Minimum Norm Interpolation via The Local Theory of Banach Spaces: The Role of Gaussianity
Gil Kur, Reese Pathak
We study minimum-norm interpolation (MNI) in overparameterized linear regression with isotropic Gaussian covariates, in settings where the MNI has no closed-form formula. Whereas m…
Sharp Risk Bounds for Early-Stopping in Gaussian Linear Regression
Tobias Wegel, Gil Kur, Patrick Rebeschini
We study early-stopped mirror descent (ESMD) for high-dimensional Gaussian linear regression over arbitrary convex bodies and design matrices, where the task is to minimize the in-…
Minimum Norm Interpolation via the Local Theory of Banach Spaces: The Role of -Uniform Convexity
Gil Kur, Pierre Bizeul
The minimum-norm interpolator (MNI) framework has recently attracted considerable attention as a tool for understanding generalization in overparameterized models, such as neural n…
Specialization after Generalization: Towards Understanding Test-Time Training in Foundation Models
Jonas Hübotter, Patrik Wolf, Alexander Shevchenko +3
Recent empirical studies have explored the idea of continuing to train a model at test-time for a given task, known as test-time training (TTT), and have found it to yield signific…
Revisiting Knowledge Distillation: The Hidden Role of Dataset Size
Giulia Lanzillotta, Felix Sarnthein, Gil Kur +2
The concept of knowledge distillation (KD) describes the training of a student model from a teacher model and is a widely adopted technique in deep learning. However, it is still n…