2k citations · 5.5k across the 40 of their papers we have counts for
49 papers
Subtraction Gets You More: Gap-Aware Retrieval for Multimodal Multi-Hop QA
Sunah O, Jay-Yoon Lee
In multimodal multi-hop question answering, we focus on the initial retrieval stage via two distinct tasks: (1) evidence set completion, retrieving missing evidence given context,…
Efficient Universal Perception Encoder
Chenchen Zhu, Saksham Suri, Cijo Jose +8
Running AI models on smart edge devices can unlock versatile user experiences, but presents challenges due to limited compute and the need to handle multiple tasks simultaneously.…
Disentangling the Factors of Convergence between Brains and Computer Vision Models
Joséphine Raugel, Marc Szafraniec, Huy V. Vo +5
Many AI models trained on natural images develop representations that resemble those of the human brain. However, the factors that drive this brain-model similarity remain poorly u…
DINOv3
Oriane Siméoni, Huy V. Vo, Maximilian Seitzer +23
Self-supervised learning holds the promise of eliminating the need for manual data annotation, enabling models to scale effortlessly to massive datasets and larger architectures. B…
Back to the Features: DINO as a Foundation for Video World Models
Federico Baldassarre, Marc Szafraniec, Basile Terver +6
We present DINO-world, a powerful generalist video world model trained to predict future frames in the latent space of DINOv2. By leveraging a pre-trained image encoder and trainin…
V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planning
Mido Assran, Adrien Bardes, David Fan +27
A major challenge for modern AI is to learn to understand the world and learn to act largely by observation. This paper explores a self-supervised approach that combines internet-s…
Cluster and Predict Latent Patches for Improved Masked Image Modeling
Timothée Darcet, Federico Baldassarre, Maxime Oquab +2
Masked Image Modeling (MIM) offers a promising approach to self-supervised representation learning, however existing MIM models still lag behind the state-of-the-art. In this paper…
DINOv2 Meets Text: A Unified Framework for Image- and Pixel-Level Vision-Language Alignment
Cijo Jose, Théo Moutakanni, Dahyun Kang +11
Self-supervised visual foundation models produce powerful embeddings that achieve remarkable performance on a wide range of downstream tasks. However, unlike vision-language models…
Automatic Data Curation for Self-Supervised Learning: A Clustering-Based Approach
Huy V. Vo, Vasil Khalidov, Timothée Darcet +10
Self-supervised features are the cornerstone of modern machine learning systems. They are typically pre-trained on data collections whose construction and curation typically requir…
You Don't Need Domain-Specific Data Augmentations When Scaling Self-Supervised Learning
Théo Moutakanni, Maxime Oquab, Marc Szafraniec +2
Self-Supervised learning (SSL) with Joint-Embedding Architectures (JEA) has led to outstanding performances. All instantiations of this paradigm were trained using strong and well-…
Advancing human-centric AI for robust X-ray analysis through holistic self-supervised learning
Théo Moutakanni, Piotr Bojanowski, Guillaume Chassagnon +7
AI Foundation models are gaining traction in various applications, including medical fields like radiology. However, medical foundation models are often tested on limited tasks, le…
Vision Transformers Need Registers
Timothée Darcet, Maxime Oquab, Julien Mairal +1
Transformers have recently emerged as a powerful tool for learning visual representations. In this paper, we identify and characterize artifacts in feature maps of both supervised…
Think Before You Act: Unified Policy for Interleaving Language Reasoning with Actions
Lina Mezghani, Piotr Bojanowski, Karteek Alahari +1
The success of transformer models trained with a language modeling objective brings a promising opportunity to the reinforcement learning framework. Decision Transformer is a step…
DINOv2: Learning Robust Visual Features without Supervision
Maxime Oquab, Timothée Darcet, Théo Moutakanni +23
The recent breakthroughs in natural language processing for model pretraining on large quantities of data have opened the way for similar foundation models in computer vision. Thes…
Learning Goal-Conditioned Policies Offline with Self-Supervised Reward Shaping
Lina Mezghani, Sainbayar Sukhbaatar, Piotr Bojanowski +2
Developing agents that can execute multiple skills by learning from pre-collected datasets is an important problem in robotics, where online interaction with the environment is ext…
Self-Supervised Learning from Images with a Joint-Embedding Predictive Architecture
Mahmoud Assran, Quentin Duval, Ishan Misra +5
This paper demonstrates an approach for learning highly semantic image representations without relying on hand-crafted data-augmentations. We introduce the Image-based Joint-Embedd…
Co-training Submodels for Visual Recognition
Hugo Touvron, Matthieu Cord, Maxime Oquab +3
We introduce submodel co-training, a regularization method related to co-training, self-distillation and stochastic depth. Given a neural network to be trained, for each sample we…
The Hidden Uniform Cluster Prior in Self-Supervised Learning
Mahmoud Assran, Randall Balestriero, Quentin Duval +6
A successful paradigm in representation learning is to perform self-supervised pretraining using tasks based on mini-batch statistics (e.g., SimCLR, VICReg, SwAV, MSN). We show tha…
Walk the Random Walk: Learning to Discover and Reach Goals Without Supervision
Lina Mezghani, Sainbayar Sukhbaatar, Piotr Bojanowski +1
Learning a diverse set of skills by interacting with an environment without any external supervision is an important challenge. In particular, obtaining a goal-conditioned agent th…
Masked Siamese Networks for Label-Efficient Learning
Mahmoud Assran, Mathilde Caron, Ishan Misra +6
We propose Masked Siamese Networks (MSN), a self-supervised learning framework for learning image representations. Our approach matches the representation of an image view containi…
Vision Models Are More Robust And Fair When Pretrained On Uncurated Images Without Supervision
Priya Goyal, Quentin Duval, Isaac Seessel +5
Discriminative self-supervised learning allows training models on any random group of internet images, and possibly recover salient information that helps differentiate between the…
Augmenting Convolutional networks with attention-based aggregation
Hugo Touvron, Matthieu Cord, Alaaeldin El-Nouby +4
We show how to augment any convolutional network with an attention-based global map to achieve non-local reasoning. We replace the final average pooling by an attention-based aggre…
Unsupervised Dense Information Retrieval with Contrastive Learning
Gautier Izacard, Mathilde Caron, Lucas Hosseini +4
Recently, information retrieval has seen the emergence of dense retrievers, using neural networks, as an alternative to classical sparse methods based on term-frequency. These mode…
XCiT: Cross-Covariance Image Transformers
Alaaeldin El-Nouby, Hugo Touvron, Mathilde Caron +8
Following their success in natural language processing, transformers have recently shown much promise for computer vision. The self-attention operation underlying transformers yiel…