6 citations · 19 across the 14 of their papers we have counts for
15 papers
Q-PACE: Dynamic Precision Allocation for Quantization-Aware Training
Alexandra Volkova, Matin Ansaripour, Erik Schultheis +2
Quantization-aware training (QAT) leverages lower-precision arithmetic to reduce the cost of LLM deployment, but aggressive quantization degrades final model performance. A common…
Towards Robust Scaling Laws for Optimizers
Alexandra Volkova, Mher Safaryan, Christoph H. Lampert +1
The quality of Large Language Model (LLM) pretraining depends on multiple factors, including the compute budget and the choice of optimization algorithm. Empirical scaling laws are…
Differentially Private Federated -Means Clustering with Server-Side Data
Jonathan Scott, Christoph H. Lampert, David Saulpic
Clustering is a cornerstone of data analysis that is particularly suited to identifying coherent subgroups or substructures in unlabeled data, as are generated continuously in larg…
ELSA: Partial Weight Freezing for Overhead-Free Sparse Network Deployment
Paniz Halvachi, Alexandra Peste, Dan Alistarh +1
We present ELSA, a practical solution for creating deep networks that can easily be deployed at different levels of sparsity. The core idea is to embed one or more sparse networks…
1-Lipschitz Neural Networks are more expressive with N-Activations
Bernd Prach, Christoph H. Lampert
A crucial property for achieving secure, trustworthy and interpretable deep learning systems is their robustness: small changes to a system's inputs should not result in large chan…
PeFLL: Personalized Federated Learning by Learning to Learn
Jonathan Scott, Hossein Zakerinia, Christoph H. Lampert
We present PeFLL, a new personalized federated learning algorithm that improves over the state-of-the-art in three aspects: 1) it produces more accurate models, especially in the l…
Deep Neural Collapse Is Provably Optimal for the Deep Unconstrained Features Model
Peter Súkeník, Marco Mondelli, Christoph Lampert
Neural collapse (NC) refers to the surprising structure of the last layer of deep neural networks in the terminal phase of gradient descent training. Recently, an increasing amount…
Predicting the Geolocation of Tweets Using transformer models on Customized Data
Kateryna Lutsai, Christoph H. Lampert
This research is aimed to solve the tweet/user geolocation prediction task and provide a flexible methodology for the geotagging of textual big data. The suggested approach impleme…
Cross-client Label Propagation for Transductive and Semi-Supervised Federated Learning
Jonathan Scott, Michelle Yeo, Christoph H. Lampert
We present Cross-Client Label Propagation(XCLP), a new method for transductive federated learning. XCLP estimates a data graph jointly from the data of multiple clients and compute…
Generalization In Multi-Objective Machine Learning
Peter Súkeník, Christoph H. Lampert
Modern machine learning tasks often require considering not just one but multiple objectives. For example, besides the prediction quality, this could be the efficiency, robustness…
Almost-Orthogonal Layers for Efficient General-Purpose Lipschitz Networks
Bernd Prach, Christoph H. Lampert
It is a highly desirable property for deep networks to be robust against small input changes. One popular way to achieve this property is by designing networks with a small Lipschi…
CrAM: A Compression-Aware Minimizer
Alexandra Peste, Adrian Vladu, Eldar Kurtic +2
Deep neural networks (DNNs) often have to be compressed, via pruning and/or quantization, before they can be deployed in practical settings. In this work we propose a new compressi…
Lightweight Conditional Model Extrapolation for Streaming Data under Class-Prior Shift
Paulina Tomaszewska, Christoph H. Lampert
We introduce LIMES, a new method for learning with non-stationary streaming data, inspired by the recent success of meta-learning. The main idea is not to attempt to learn a single…
SSSE: Efficiently Erasing Samples from Trained Machine Learning Models
Alexandra Peste, Dan Alistarh, Christoph H. Lampert
The availability of large amounts of user-provided data has been key to the success of machine learning for many real-world tasks. Recently, an increasing awareness has emerged tha…
Fairness-Aware PAC Learning from Corrupted Data
Nikola Konstantinov, Christoph H. Lampert
Addressing fairness concerns about machine learning models is a crucial step towards their long-term adoption in real-world automated systems. While many approaches have been devel…