4 citations · 4 across the 11 of their papers we have counts for
12 papers
Bigger Text Encoders Can Hurt CLIP Zero-Shot Performance
Samir Char, Carles Domingo-Enrich, Randall Balestriero
Contrastive Language-Image Pretraining (CLIP) is a building block of many machine learning applications. Scaling laws have guided resource allocation for large-scale training, yet…
LeVLJEPA: End-to-End Vision-Language Pretraining Without Negatives
Lukas Kuhn, Giuseppe Serra, Randall Balestriero +1
Vision-language pretraining remains dominated by contrastive objectives, whereas vision-only self-supervised learning has largely adopted non-contrastive methods. At the same time,…
PrAg-PO: Prompt Augmented Policy Optimization for Robust and Diverse Mathematical Reasoning
Wenquan Lu, Hai Huang, Enqi Liu +1
Reinforcement learning algorithms such as group-relative policy optimization (GRPO) have shown strong potential for improving the mathematical reasoning capabilities of large langu…
Into the Rabbit Hull: From Task-Relevant Concepts in DINO to Minkowski Geometry
Thomas Fel, Binxu Wang, Michael A. Lepori +8
DINOv2 is routinely deployed to recognize objects, scenes, and actions; yet the nature of what it perceives remains unknown. As a working baseline, we adopt the Linear Representati…
DIET-CP: Lightweight and Data Efficient Self Supervised Continued Pretraining
Bryan Rodas, Natalie Montesino, Jakob Ambsdorf +2
Continued pretraining offers a promising solution for adapting foundation models to a new target domain. However, in specialized domains, available datasets are often very small, l…
Your Attention Matters: to Improve Model Robustness to Noise and Spurious Correlations
Camilo Tamayo-Rousseau, Yunjia Zhao, Yiqun Zhang +1
Self-attention mechanisms are foundational to Transformer architectures, supporting their impressive success in a wide range of tasks. While there are many self-attention variants,…
Task Priors: Enhancing Model Evaluation by Considering the Entire Space of Downstream Tasks
Niket Patel, Randall Balestriero
The grand goal of AI research, and particularly Self Supervised Learning (SSL), is to produce systems that can successfully solve any possible task. In contrast, current evaluation…
FastDINOv2: Frequency Based Curriculum Learning Improves Robustness and Training Speed
Jiaqi Zhang, Juntuo Wang, Zhixin Sun +2
Large-scale vision foundation models such as DINOv2 boast impressive performances by leveraging massive architectures and training datasets. But numerous scenarios require practiti…
General Methods Make Great Domain-specific Foundation Models: A Case-study on Fetal Ultrasound
Jakob Ambsdorf, Asbjørn Munk, Sebastian Llambias +6
With access to large-scale, unlabeled medical datasets, researchers are confronted with two questions: Should they attempt to pretrain a custom foundation model on this medical dat…
LoRA Users Beware: A Few Spurious Tokens Can Manipulate Your Finetuned Model
Marcel Mateos Salles, Praney Goyal, Pradyut Sekhsaria +2
Large Language Models (LLMs) are commonly finetuned for a variety of use cases and domains. A common approach is to leverage Low-Rank Adaptation (LoRA) -- known to provide strong p…
Next Token Perception Score: Analytical Assessment of your LLM Perception Skills
Yu-Ang Cheng, Leyang Hu, Hai Huang +1
Autoregressive pretraining has become the de facto paradigm for learning general-purpose representations in large language models (LLMs). However, linear probe performance across d…
SplInterp: Improving our Understanding and Training of Sparse Autoencoders
Jeremy Budd, Javier Ideami, Benjamin Macdowall Rynne +2
Sparse autoencoders (SAEs) have received considerable recent attention as tools for mechanistic interpretability, showing success at extracting interpretable features even from ver…