6 papers
Local Sparsity Enables Unsupervised LLM Safety Detection
Xin Chen, Gil Kur, Alexander Shevchenko +1
Deployment-time safety methods for large language models (LLMs) are predominantly supervised and assume access to unsafe training data. Nevertheless, new attacks and harm categorie…
Minimum Norm Interpolation via The Local Theory of Banach Spaces: The Role of Gaussianity
Gil Kur, Reese Pathak
We study minimum-norm interpolation (MNI) in overparameterized linear regression with isotropic Gaussian covariates, in settings where the MNI has no closed-form formula. Whereas m…
Minimum Norm Interpolation via the Local Theory of Banach Spaces: The Role of -Uniform Convexity
Gil Kur, Pierre Bizeul
The minimum-norm interpolator (MNI) framework has recently attracted considerable attention as a tool for understanding generalization in overparameterized models, such as neural n…
Revisiting Knowledge Distillation: The Hidden Role of Dataset Size
Giulia Lanzillotta, Felix Sarnthein, Gil Kur +2
The concept of knowledge distillation (KD) describes the training of a student model from a teacher model and is a widely adopted technique in deep learning. However, it is still n…
Specialization after Generalization: Towards Understanding Test-Time Training in Foundation Models
Jonas Hübotter, Patrik Wolf, Alexander Shevchenko +3
Recent empirical studies have explored the idea of continuing to train a model at test-time for a given task, known as test-time training (TTT), and have found it to yield signific…
Sharp Risk Bounds for Early-Stopping in Gaussian Linear Regression
Tobias Wegel, Gil Kur, Patrick Rebeschini
We study early-stopped mirror descent (ESMD) for high-dimensional Gaussian linear regression over arbitrary convex bodies and design matrices, where the task is to minimize the in-…