6 citations · 17 across the 17 of their papers we have counts for
18 papers
Gemma 4 Technical Report
Gemma Team, Sherif El Abd, Vaibhav Aggarwal +320
We introduce Gemma 4, a new generation of open-weight, natively multimodal language models in the Gemma model family. Designed to advance compute efficiency and reasoning, the Gemm…
Unveiling Simplicities of Attention: Adaptive Long-Context Head Identification
Konstantin Donhauser, Charles Arnal, Mohammad Pezeshki +3
The ability to process long contexts is crucial for many natural language processing tasks, yet it remains a significant challenge. While substantial progress has been made in enha…
Efficient Randomized Experiments Using Foundation Models
Piersilvio De Bartolomeis, Javier Abad, Guanbo Wang +4
Randomized experiments are the preferred approach for evaluating the effects of interventions, but they are costly and often yield estimates with substantial uncertainty. On the ot…
Copyright-Protected Language Generation via Adaptive Model Fusion
Javier Abad, Konstantin Donhauser, Francesco Pinto +1
The risk of language models reproducing copyrighted material from their training data has led to the development of various protective measures. Among these, inference-time strateg…
Towards scientific discovery with dictionary learning: Extracting biological concepts from microscopy foundation models
Konstantin Donhauser, Kristina Ulicna, Gemma Elyse Moran +4
Sparse dictionary learning (DL) has emerged as a powerful approach to extract semantically meaningful concepts from the internals of large language models (LLMs) trained mainly in…
ViTally Consistent: Scaling Biological Representation Learning for Cell Microscopy
Kian Kenyon-Dean, Zitong Jerry Wang, John Urbanik +10
Large-scale cell microscopy screens are used in drug discovery and molecular biology research to study the effects of millions of chemical and genetic perturbations on cells. To us…
Strong Copyright Protection for Language Models via Adaptive Model Fusion
Javier Abad, Konstantin Donhauser, Francesco Pinto +1
The risk of language models unintentionally reproducing copyrighted material from their training data has led to the development of various protective measures. In this paper, we p…
Detecting critical treatment effect bias in small subgroups
Piersilvio De Bartolomeis, Javier Abad, Konstantin Donhauser +1
Randomized trials are considered the gold standard for making informed decisions in medicine, yet they often lack generalizability to the patient populations in clinical practice.…
Privacy-preserving data release leveraging optimal transport and particle gradient descent
Konstantin Donhauser, Javier Abad, Neha Hulkund +1
We present a novel approach for differentially private data synthesis of protected tabular datasets, a relevant task in highly sensitive domains such as healthcare and government.…
Hidden yet quantifiable: A lower bound for confounding strength using randomized trials
Piersilvio De Bartolomeis, Javier Abad, Konstantin Donhauser +1
In the era of fast-paced precision medicine, observational studies play a major role in properly evaluating new treatments in clinical practice. Yet, unobserved confounding can sig…
Strong inductive biases provably prevent harmless interpolation
Michael Aerni, Marco Milanta, Konstantin Donhauser +1
Classical wisdom suggests that estimators should avoid fitting noise to achieve good generalization. In contrast, modern overparameterized models can yield small test error despite…
Certified private data release for sparse Lipschitz functions
Konstantin Donhauser, Johan Lokna, Amartya Sanyal +3
As machine learning has become more relevant for everyday applications, a natural requirement is the protection of the privacy of the training data. When the relevant learning ques…
Tight bounds for maximum -margin classifiers
Stefan Stojanovic, Konstantin Donhauser, Fanny Yang
Popular iterative algorithms such as boosting methods and coordinate descent on linear models converge to the maximum -margin classifier, a.k.a. sparse hard-margin SVM, in…
Fast Rates for Noisy Interpolation Require Rethinking the Effects of Inductive Bias
Konstantin Donhauser, Nicolo Ruggeri, Stefan Stojanovic +1
Good generalization performance on high-dimensional data crucially hinges on a simple structure of the ground truth and a corresponding strong inductive bias of the estimator. Even…
Tight bounds for minimum l1-norm interpolation of noisy data
Guillaume Wang, Konstantin Donhauser, Fanny Yang
We provide matching upper and lower bounds of order for the prediction error of the minimum -norm interpolator, a.k.a. basis pursuit. Our result is tight up…
Interpolation can hurt robust generalization even when there is no noise
Konstantin Donhauser, Alexandru Ţifrea, Michael Aerni +2
Numerous recent works show that overparameterization implicitly reduces variance for min-norm interpolators and max-margin classifiers. These findings suggest that ridge regulariza…
How rotational invariance of common kernels prevents generalization in high dimensions
Konstantin Donhauser, Mingqi Wu, Fanny Yang
Kernel ridge regression is well-known to achieve minimax optimal rates in low-dimensional settings. However, its behavior in high dimensions is much less understood. Recent work es…
Efficient Smoothing of Dilated Convolutions for Image Segmentation
Thomas Ziegler, Manuel Fritsche, Lorenz Kuhn +1
Dilated Convolutions have been shown to be highly useful for the task of image segmentation. By introducing gaps into convolutional filters, they enable the use of larger receptive…