1.4k citations · 2.1k across the 21 of their papers we have counts for
22 papers
Gemma 4 Technical Report
Gemma Team, Sherif El Abd, Vaibhav Aggarwal +320
We introduce Gemma 4, a new generation of open-weight, natively multimodal language models in the Gemma model family. Designed to advance compute efficiency and reasoning, the Gemm…
Fine-Grained Distribution-Dependent Learning Curves
Olivier Bousquet, Steve Hanneke, Shay Moran +2
Learning curves plot the expected error of a learning algorithm as a function of the number of labeled samples it receives from a target distribution. They are widely used as a mea…
A Generalized Lottery Ticket Hypothesis
Ibrahim Alabdulmohsin, Larisa Markeeva, Daniel Keysers +1
We introduce a generalization to the lottery ticket hypothesis in which the notion of "sparsity" is relaxed by choosing an arbitrary basis in the space of parameters. We present ev…
MLP-Mixer: An all-MLP Architecture for Vision
Ilya Tolstikhin, Neil Houlsby, Alexander Kolesnikov +9
Convolutional Neural Networks (CNNs) are the go-to model for computer vision. Recently, attention-based networks, such as the Vision Transformer, have also become popular. In this…
What Do Neural Networks Learn When Trained With Random Labels?
Hartmut Maennel, Ibrahim Alabdulmohsin, Ilya Tolstikhin +4
We study deep neural networks (DNNs) trained on natural image data with entirely random labels. Despite its popularity in the literature, where it is often used to study memorizati…
Predicting Neural Network Accuracy from Weights
Thomas Unterthiner, Daniel Keysers, Sylvain Gelly +2
We show experimentally that the accuracy of a trained neural network can be predicted surprisingly well by looking only at its weights, without evaluating it on input data. We moti…
When can unlabeled data improve the learning rate?
Christina Göpfert, Shai Ben-David, Olivier Bousquet +3
In semi-supervised classification, one is given access both to labeled and unlabeled data. As unlabeled data is typically cheaper to acquire than labeled data, this setup becomes a…
Practical and Consistent Estimation of f-Divergences
Paul K. Rubenstein, Olivier Bousquet, Josip Djolonga +2
The estimation of an f-divergence between two probability distributions based on samples is a fundamental problem in statistics and machine learning. Most works study this problem…
GeNet: Deep Representations for Metagenomics
Mateo Rojas-Carulla, Ilya Tolstikhin, Guillermo Luque +3
We introduce GeNet, a method for shotgun metagenomic classification from raw DNA sequences that exploits the known hierarchical structure between labels for training. We provide a…
Competitive Training of Mixtures of Independent Deep Generative Models
Francesco Locatello, Damien Vincent, Ilya Tolstikhin +3
A common assumption in causal modeling posits that the data is generated by a set of independent mechanisms, and algorithms should aim to recover this structure. Standard unsupervi…
On the Latent Space of Wasserstein Auto-Encoders
Paul K. Rubenstein, Bernhard Schoelkopf, Ilya Tolstikhin
We study the role of latent space dimensionality in Wasserstein auto-encoders (WAEs). Through experimentation on synthetic and real datasets, we argue that random encoders should b…
Wasserstein Auto-Encoders
Ilya Tolstikhin, Olivier Bousquet, Sylvain Gelly +1
We propose the Wasserstein Auto-Encoder (WAE)---a new algorithm for building a generative model of the data distribution. WAE minimizes a penalized form of the Wasserstein distance…
Differentially Private Database Release via Kernel Mean Embeddings
Matej Balog, Ilya Tolstikhin, Bernhard Schölkopf
We lay theoretical foundations for new database release mechanisms that allow third-parties to construct consistent estimators of population statistics, while ensuring that the pri…
Probabilistic Active Learning of Functions in Structural Causal Models
Paul K. Rubenstein, Ilya Tolstikhin, Philipp Hennig +1
We consider the problem of learning the functions computing children from parents in a Structural Causal Model once the underlying causal graph has been identified. This is in some…
From optimal transport to generative modeling: the VEGAN cookbook
Olivier Bousquet, Sylvain Gelly, Ilya Tolstikhin +2
We study unsupervised generative modeling in terms of the optimal transport (OT) problem between true (but unknown) data distribution and the latent variable model distributi…
AdaGAN: Boosting Generative Models
Ilya Tolstikhin, Sylvain Gelly, Olivier Bousquet +2
Generative Adversarial Networks (GAN) (Goodfellow et al., 2014) are an effective method for training generative models of complex data such as natural images. However, they are not…
Consistent Kernel Mean Estimation for Functions of Random Variables
Carl-Johann Simon-Gabriel, Adam Ścibior, Ilya Tolstikhin +1
We provide a theoretical foundation for non-parametric estimation of functions of random variables using kernel mean embeddings. We show that for any continuous function , consi…
Minimax Lower Bounds for Realizable Transductive Classification
Ilya Tolstikhin, David Lopez-Paz
Transductive learning considers a training set of labeled samples and a test set of unlabeled samples, with the goal of best labeling that particular test set. Conversely,…
Minimax Estimation of Kernel Mean Embeddings
Ilya Tolstikhin, Bharath Sriperumbudur, Krikamol Muandet
In this paper, we study the minimax estimation of the Bochner integral also called as the kernel mean embedding, based on random s…
Permutational Rademacher Complexity: a New Complexity Measure for Transductive Learning
Ilya Tolstikhin, Nikita Zhivotovskiy, Gilles Blanchard
Transductive learning considers situations when a learner observes labelled training points and unlabelled test points with the final goal of giving correct answers for the…
Towards a Learning Theory of Cause-Effect Inference
David Lopez-Paz, Krikamol Muandet, Bernhard Schölkopf +1
We pose causal inference as the problem of learning to classify probability distributions. In particular, we assume access to a collection , where each …
Localized Complexities for Transductive Learning
Ilya Tolstikhin, Gilles Blanchard, Marius Kloft
We show two novel concentration inequalities for suprema of empirical processes when sampling without replacement, which both take the variance of the functions into account. While…