137 citations · 348 across the 31 of their papers we have counts for
33 papers
Where Does the Signal Live? A Web Data Recipe for Medical Encoder Pretraining
Bofeng Huang, Jacques Sun, Diane Bouchacourt +2
Web data curation has been widely studied for decoder Large Language Model (LLM) pretraining. Encoders for dense-terminology domains such as medicine, by contrast, are pretrained o…
Reassessing the Validity of Spurious Correlations Benchmarks
Samuel J. Bell, Diane Bouchacourt, Levent Sagun
Neural networks can fail when the data contains spurious correlations. To understand this phenomenon, researchers have proposed numerous spurious correlations benchmarks upon which…
UniBench: Visual Reasoning Requires Rethinking Vision-Language Beyond Scaling
Haider Al-Tahan, Quentin Garrido, Randall Balestriero +3
Significant research efforts have been made to scale and improve vision-language model (VLM) training approaches. Yet, with an ever-growing number of benchmarks, researchers are ta…
-Sample Contrastive Loss: Improving Contrastive Learning with Sample Similarity Graphs
Vlad Sobal, Mark Ibrahim, Randall Balestriero +5
Learning good representations involves capturing the diverse ways in which data samples relate. Contrastive loss - an objective matching related samples - underlies methods from se…
The Factorization Curse: Which Tokens You Predict Underlie the Reversal Curse and More
Ouail Kitouni, Niklas Nolte, Diane Bouchacourt +3
Today's best language models still struggle with hallucinations: factually incorrect generations, which impede their ability to reliably retrieve information seen during training.…
An Introduction to Vision-Language Modeling
Florian Bordes, Richard Yuanzhe Pang, Anurag Ajay +38
Following the recent popularity of Large Language Models (LLMs), several attempts have been made to extend them to the visual domain. From having a visual assistant that could guid…
Embracing Diversity: Interpretable Zero-shot classification beyond one vector per class
Mazda Moayeri, Michael Rabbat, Mark Ibrahim +1
Vision-language models enable open-world classification of objects without the need for any retraining. While this zero-shot paradigm marks a significant advance, even today's best…
Understanding the Detrimental Class-level Effects of Data Augmentation
Polina Kirichenko, Mark Ibrahim, Randall Balestriero +4
Data augmentation (DA) encodes invariance and provides implicit regularization critical to a model's performance in image classification tasks. However, while DA improves average a…
Self-Supervised Disentanglement by Leveraging Structure in Data Augmentations
Cian Eastwood, Julius von Kügelgen, Linus Ericsson +4
Self-supervised representation learning often uses data augmentations to induce some invariance to "style" attributes of the data. However, with downstream tasks generally unknown…
Discovering environments with XRM
Mohammad Pezeshki, Diane Bouchacourt, Mark Ibrahim +3
Environment annotations are essential for the success of many out-of-distribution (OOD) generalization methods. Unfortunately, these are costly to obtain and often limited by human…
PUG: Photorealistic and Semantically Controllable Synthetic Data for Representation Learning
Florian Bordes, Shashank Shekhar, Mark Ibrahim +3
Synthetic image datasets offer unmatched advantages for designing and evaluating deep neural networks: they make it possible to (i) render as many data samples as needed, (ii) prec…
Does Progress On Object Recognition Benchmarks Improve Real-World Generalization?
Megan Richards, Polina Kirichenko, Diane Bouchacourt +1
For more than a decade, researchers have measured progress in object recognition on ImageNet-based generalization benchmarks such as ImageNet-A, -C, and -R. Recent advances in foun…
Birth of a Transformer: A Memory Viewpoint
Alberto Bietti, Vivien Cabannes, Diane Bouchacourt +2
Large language models based on transformers have achieved great empirical successes. However, as they are deployed more widely, there is a growing need to better understand their i…
Pinpointing Why Object Recognition Performance Degrades Across Income Levels and Geographies
Laura Gustafson, Megan Richards, Melissa Hall +3
Despite impressive advances in object-recognition, deep learning systems' performance degrades significantly across geographies and lower income levels raising pressing concerns of…
ImageNet-X: Understanding Model Mistakes with Factor of Variation Annotations
Badr Youbi Idrissi, Diane Bouchacourt, Randall Balestriero +7
Deep learning vision systems are widely deployed across applications where reliability is critical. However, even today's best models can fail to recognize an object when its pose,…
The Robustness Limits of SoTA Vision Models to Natural Variation
Mark Ibrahim, Quentin Garrido, Ari Morcos +1
Recent state-of-the-art vision models introduced new architectures, learning paradigms, and larger pretraining data, leading to impressive performance on tasks such as classificati…
Robust Self-Supervised Learning with Lie Groups
Mark Ibrahim, Diane Bouchacourt, Ari Morcos
Deep learning has led to remarkable advances in computer vision. Even so, today's best models are brittle when presented with variations that differ even slightly from those seen d…
Disentanglement of Correlated Factors via Hausdorff Factorized Support
Karsten Roth, Mark Ibrahim, Zeynep Akata +2
A grand goal in deep learning research is to learn representations capable of generalizing across distribution shifts. Disentanglement is one promising direction aimed at aligning…
Measuring and signing fairness as performance under multiple stakeholder distributions
David Lopez-Paz, Diane Bouchacourt, Levent Sagun +1
As learning machines increase their influence on decisions concerning human lives, analyzing their fairness properties becomes a subject of central importance. Yet, our best tools…
Grounding inductive biases in natural images:invariance stems from variations in data
Diane Bouchacourt, Mark Ibrahim, Ari S. Morcos
To perform well on unseen and potentially out-of-distribution samples, it is desirable for machine learning models to have a predictable response with respect to transformations af…
Addressing the Topological Defects of Disentanglement via Distributed Operators
Diane Bouchacourt, Mark Ibrahim, Stéphane Deny
A core challenge in Machine Learning is to learn to disentangle natural factors of variation in data (e.g. object shape vs. pose). A popular approach to disentanglement consists in…
Think before you act: A simple baseline for compositional generalization
Christina Heinze-Deml, Diane Bouchacourt
Contrarily to humans who have the ability to recombine familiar expressions to create novel ones, modern neural networks struggle to do so. This has been emphasized recently with t…
Compositionality and Generalization in Emergent Languages
Rahma Chaabouni, Eugene Kharitonov, Diane Bouchacourt +2
Natural language allows us to refer to novel composite concepts by combining expressions denoting their parts according to systematic rules, a property known as \emph{compositional…
A Benchmark for Systematic Generalization in Grounded Language Understanding
Laura Ruis, Jacob Andreas, Marco Baroni +2
Humans easily interpret expressions that describe unfamiliar situations composed from familiar parts ("greet the pink brontosaurus by the ferris wheel"). Modern neural networks, by…