1.4k citations · 2.6k across the 79 of their papers we have counts for
97 papers
JAFAR: Jack up Any Feature at Any Resolution
Paul Couairon, Loick Chambon, Louis Serrano +3
Foundation Vision Encoders have become essential for a wide range of dense vision tasks. However, their low-resolution spatial feature outputs necessitate feature upsampling to pro…
Beyond Task Performance: Evaluating and Reducing the Flaws of Large Multimodal Models with In-Context Learning
Mustafa Shukor, Alexandre Rame, Corentin Dancette +1
Following the success of Large Language Models (LLMs), Large Multimodal Models (LMMs), such as the Flamingo model and its subsequent competitors, have started to emerge as natural…
UnIVAL: Unified Model for Image, Video, Audio and Language Tasks
Mustafa Shukor, Corentin Dancette, Alexandre Rame +1
Large Language Models (LLMs) have made the ambitious quest for generalist agents significantly far from being a fantasy. A key hurdle for building such general models is the divers…
MOCA: Self-supervised Representation Learning by Predicting Masked Online Codebook Assignments
Spyros Gidaris, Andrei Bursuc, Oriane Simeoni +4
Self-supervised learning can be used for mitigating the greedy needs of Vision Transformer networks for very large fully-annotated datasets. Different classes of self-supervised le…
Zero-shot spatial layout conditioning for text-to-image diffusion models
Guillaume Couairon, Marlène Careil, Matthieu Cord +2
Large-scale text-to-image diffusion models have significantly improved the state of the art in generative image modelling and allow for an intuitive and powerful user interface to…
Improving Selective Visual Question Answering by Learning from Your Peers
Corentin Dancette, Spencer Whitehead, Rishabh Maheshwary +5
Despite advances in Visual Question Answering (VQA), the ability of models to assess their own correctness remains underexplored. Recent work has shown that VQA models, out-of-the-…
OBELICS: An Open Web-Scale Filtered Dataset of Interleaved Image-Text Documents
Hugo Laurençon, Lucile Saulnier, Léo Tronchon +9
Large multimodal models trained on natural documents, which interleave images and text, outperform models trained on image-text pairs on various multimodal benchmarks. However, the…
Towards Motion Forecasting with Real-World Perception Inputs: Are End-to-End Approaches Competitive?
Yihong Xu, Loïck Chambon, Éloi Zablocki +4
Motion forecasting is crucial in enabling autonomous vehicles to anticipate the future trajectories of surrounding agents. To do so, it requires solving mapping, detection, trackin…
Rewarded soups: towards Pareto-optimal alignment by interpolating weights fine-tuned on diverse rewards
Alexandre Ramé, Guillaume Couairon, Mustafa Shukor +4
Foundation models are first pre-trained on vast unsupervised datasets and then fine-tuned on labeled data. Reinforcement learning, notably from human feedback (RLHF), can further a…
eP-ALM: Efficient Perceptual Augmentation of Language Models
Mustafa Shukor, Corentin Dancette, Matthieu Cord
Large Language Models (LLMs) have so far impressed the world, with unprecedented capabilities that emerge in models at large scales. On the vision side, transformer models (i.e., V…
PowerQuant: Automorphism Search for Non-Uniform Quantization
Edouard Yvinec, Arnaud Dapogny, Matthieu Cord +1
Deep neural networks (DNNs) are nowadays ubiquitous in many domains such as computer vision. However, due to their high latency, the deployment of DNNs hinges on the development of…
Co-training Submodels for Visual Recognition
Hugo Touvron, Matthieu Cord, Maxime Oquab +3
We introduce submodel co-training, a regularization method related to co-training, self-distillation and stochastic depth. Given a neural network to be trained, for each sample we…
Model Ratatouille: Recycling Diverse Models for Out-of-Distribution Generalization
Alexandre Ramé, Kartik Ahuja, Jianyu Zhang +3
Foundation models are redefining how AI systems are built. Practitioners now follow a standard procedure to build their machine learning solutions: from a pre-trained foundation mo…
Vision and Structured-Language Pretraining for Cross-Modal Food Retrieval
Mustafa Shukor, Nicolas Thome, Matthieu Cord
Vision-Language Pretraining (VLP) and Foundation models have been the go-to recipe for achieving SoTA performance on general benchmarks. However, leveraging these powerful techniqu…
CoMFormer: Continual Learning in Semantic and Panoptic Segmentation
Fabio Cermelli, Matthieu Cord, Arthur Douillard
Continual learning for segmentation has recently seen increasing interest. However, all previous works focus on narrow semantic segmentation and disregard panoptic segmentation, an…
OCTET: Object-aware Counterfactual Explanations
Mehdi Zemni, Mickaël Chen, Éloi Zablocki +3
Nowadays, deep vision models are being widely deployed in safety-critical applications, e.g., autonomous driving, and explainability of such models is becoming a pressing concern.…
DiffEdit: Diffusion-based semantic image editing with mask guidance
Guillaume Couairon, Jakob Verbeek, Holger Schwenk +1
Image generation has recently seen tremendous advances, with diffusion models allowing to synthesize convincing images for a large variety of text prompts. In this article, we prop…
Efficient Vision-Language Pretraining with Visual Concepts and Hierarchical Alignment
Mustafa Shukor, Guillaume Couairon, Matthieu Cord
Vision and Language Pretraining has become the prevalent approach for tackling multimodal downstream tasks. The current trend is to move towards ever larger models and pretraining…
SInGE: Sparsity via Integrated Gradients Estimation of Neuron Relevance
Edouard Yvinec, Arnaud Dapogny, Matthieu Cord +1
The leap in performance in state-of-the-art computer vision methods is attributed to the development of deep neural networks. However it often comes at a computational price which…
LaRa: Latents and Rays for Multi-Camera Bird's-Eye-View Semantic Segmentation
Florent Bartoccioni, Éloi Zablocki, Andrei Bursuc +3
Recent works in autonomous driving have widely adopted the bird's-eye-view (BEV) semantic map as an intermediate representation of the world. Online prediction of these BEV maps in…
Swapping Semantic Contents for Mixing Images
Rémy Sun, Clément Masson, Gilles Hénaff +2
Deep architecture have proven capable of solving many tasks provided a sufficient amount of labeled data. In fact, the amount of available labeled data has become the principal bot…
Towards efficient feature sharing in MIMO architectures
Rémy Sun, Alexandre Ramé, Clément Masson +2
Multi-input multi-output architectures propose to train multiple subnetworks within one base network and then average the subnetwork predictions to benefit from ensembling for free…
Dynamic Query Selection for Fast Visual Perceiver
Corentin Dancette, Matthieu Cord
Transformers have been matching deep convolutional networks for vision architectures in recent works. Most work is focused on getting the best results on large-scale benchmarks, an…
Diverse Weight Averaging for Out-of-Distribution Generalization
Alexandre Ramé, Matthieu Kirchmeyer, Thibaud Rahier +3
Standard neural networks struggle to generalize under distribution shifts in computer vision. Fortunately, combining multiple networks can consistently improve out-of-distribution…