55 citations · 135 across the 10 of their papers we have counts for
8 papers · 1 filter
AlpacaFarm: A Simulation Framework for Methods that Learn from Human Feedback
Yann Dubois, Xuechen Li, Rohan Taori +6
Large language models (LLMs) such as ChatGPT have seen widespread adoption due to their strong instruction-following abilities. Developing these LLMs involves a complex yet poorly…
Evaluating Self-Supervised Learning via Risk Decomposition
Yann Dubois, Tatsunori Hashimoto, Percy Liang
Self-supervised learning (SSL) pipelines differ in many design choices such as the architecture, augmentations, or pretraining data. Yet SSL is typically evaluated using a single m…
Improving Self-Supervised Learning by Characterizing Idealized Representations
Yann Dubois, Tatsunori Hashimoto, Stefano Ermon +1
Despite the empirical successes of self-supervised learning (SSL) methods, it is unclear what characteristics of their representations lead to high downstream accuracies. In this w…
Learning Instance-Specific Augmentations by Capturing Local Invariances
Ning Miao, Tom Rainforth, Emile Mathieu +4
We introduce InstaAug, a method for automatically learning input-specific augmentations from data. Previous methods for learning augmentations have typically assumed independence b…
Optimal Representations for Covariate Shift
Yangjun Ruan, Yann Dubois, Chris J. Maddison
Machine learning systems often experience a distribution shift between training and testing. In this paper, we introduce a simple variational objective whose optima are exactly the…
Lossy Compression for Lossless Prediction
Yann Dubois, Benjamin Bloem-Reddy, Karen Ullrich +1
Most data is automatically collected and only ever "seen" by algorithms. Yet, data compressors preserve perceptual fidelity rather than just the information needed by algorithms pe…
Learning Optimal Representations with the Decodable Information Bottleneck
Yann Dubois, Douwe Kiela, David J. Schwab +1
We address the question of characterizing and finding optimal representations for supervised learning. Traditionally, this question has been tackled using the Information Bottlenec…
Location Attention for Extrapolation to Longer Sequences
Yann Dubois, Gautier Dagan, Dieuwke Hupkes +1
Neural networks are surprisingly good at interpolating and perform remarkably well when the training set examples resemble those in the test set. However, they are often unable to…