1.4k citations · 2.6k across the 79 of their papers we have counts for
10 papers · 1 filter
Beyond Task Performance: Evaluating and Reducing the Flaws of Large Multimodal Models with In-Context Learning
Mustafa Shukor, Alexandre Rame, Corentin Dancette +1
Following the success of Large Language Models (LLMs), Large Multimodal Models (LMMs), such as the Flamingo model and its subsequent competitors, have started to emerge as natural…
UnIVAL: Unified Model for Image, Video, Audio and Language Tasks
Mustafa Shukor, Corentin Dancette, Alexandre Rame +1
Large Language Models (LLMs) have made the ambitious quest for generalist agents significantly far from being a fantasy. A key hurdle for building such general models is the divers…
MOCA: Self-supervised Representation Learning by Predicting Masked Online Codebook Assignments
Spyros Gidaris, Andrei Bursuc, Oriane Simeoni +4
Self-supervised learning can be used for mitigating the greedy needs of Vision Transformer networks for very large fully-annotated datasets. Different classes of self-supervised le…
Zero-shot spatial layout conditioning for text-to-image diffusion models
Guillaume Couairon, Marlène Careil, Matthieu Cord +2
Large-scale text-to-image diffusion models have significantly improved the state of the art in generative image modelling and allow for an intuitive and powerful user interface to…
Improving Selective Visual Question Answering by Learning from Your Peers
Corentin Dancette, Spencer Whitehead, Rishabh Maheshwary +5
Despite advances in Visual Question Answering (VQA), the ability of models to assess their own correctness remains underexplored. Recent work has shown that VQA models, out-of-the-…
OBELICS: An Open Web-Scale Filtered Dataset of Interleaved Image-Text Documents
Hugo Laurençon, Lucile Saulnier, Léo Tronchon +9
Large multimodal models trained on natural documents, which interleave images and text, outperform models trained on image-text pairs on various multimodal benchmarks. However, the…
Towards Motion Forecasting with Real-World Perception Inputs: Are End-to-End Approaches Competitive?
Yihong Xu, Loïck Chambon, Éloi Zablocki +4
Motion forecasting is crucial in enabling autonomous vehicles to anticipate the future trajectories of surrounding agents. To do so, it requires solving mapping, detection, trackin…
Rewarded soups: towards Pareto-optimal alignment by interpolating weights fine-tuned on diverse rewards
Alexandre Ramé, Guillaume Couairon, Mustafa Shukor +4
Foundation models are first pre-trained on vast unsupervised datasets and then fine-tuned on labeled data. Reinforcement learning, notably from human feedback (RLHF), can further a…
eP-ALM: Efficient Perceptual Augmentation of Language Models
Mustafa Shukor, Corentin Dancette, Matthieu Cord
Large Language Models (LLMs) have so far impressed the world, with unprecedented capabilities that emerge in models at large scales. On the vision side, transformer models (i.e., V…
PowerQuant: Automorphism Search for Non-Uniform Quantization
Edouard Yvinec, Arnaud Dapogny, Matthieu Cord +1
Deep neural networks (DNNs) are nowadays ubiquitous in many domains such as computer vision. However, due to their high latency, the deployment of DNNs hinges on the development of…