266 citations · 330 across the 12 of their papers we have counts for
13 papers
Flow Matching in Feature Space for Stochastic World Modeling
Francois Porcher, Nicolas Carion, Karteek Alahari +1
World modeling requires forecasting uncertain futures while preserving information useful for downstream perception. Existing visual world models often struggle to satisfy both goa…
SAM 3: Segment Anything with Concepts
Nicolas Carion, Laura Gustafson, Yuan-Ting Hu +35
We present Segment Anything Model (SAM) 3, a unified model that detects, segments, and tracks objects in images and videos based on concept prompts, which we define as either short…
SAM 2: Segment Anything in Images and Videos
Nikhila Ravi, Valentin Gabeur, Yuan-Ting Hu +15
We present Segment Anything Model 2 (SAM 2), a foundation model towards solving promptable visual segmentation in images and videos. We build a data engine, which improves model an…
DIG In: Evaluating Disparities in Image Generations with Indicators for Geographic Diversity
Melissa Hall, Candace Ross, Adina Williams +3
The unprecedented photorealistic results achieved by recent text-to-image generative systems and their increasing use as plug-and-play content creation solutions make it crucial to…
GeneCIS: A Benchmark for General Conditional Image Similarity
Sagar Vaze, Nicolas Carion, Ishan Misra
We argue that there are many notions of 'similarity' and that models, like humans, should be able to adapt to these dynamically. This contrasts with most representation learning me…
Joint Embedding Predictive Architectures Focus on Slow Features
Vlad Sobal, Jyothir S, Siddhartha Jalagam +3
Many common methods for learning a world model for pixel-based environments use generative architectures trained with pixel-level reconstruction objectives. Recently proposed Joint…
Light-weight probing of unsupervised representations for Reinforcement Learning
Wancong Zhang, Anthony GX-Chen, Vlad Sobal +2
Unsupervised visual representation learning offers the opportunity to leverage large corpora of unlabeled trajectories to form useful visual representations, which can benefit the…
Separating the World and Ego Models for Self-Driving
Vlad Sobal, Alfredo Canziani, Nicolas Carion +2
Training self-driving systems to be robust to the long-tail of driving scenarios is a critical problem. Model-based approaches leverage simulation to emulate a wide range of scenar…
MDETR -- Modulated Detection for End-to-End Multi-Modal Understanding
Aishwarya Kamath, Mannat Singh, Yann LeCun +3
Multi-modal reasoning systems rely on a pre-trained object detector to extract regions of interest from the image. However, this crucial module is typically used as a black box, tr…
End-to-End Object Detection with Transformers
Nicolas Carion, Francisco Massa, Gabriel Synnaeve +3
We present a new method that views object detection as a direct set prediction problem. Our approach streamlines the detection pipeline, effectively removing the need for many hand…
A Structured Prediction Approach for Generalization in Cooperative Multi-Agent Reinforcement Learning
Nicolas Carion, Gabriel Synnaeve, Alessandro Lazaric +1
Effective coordination is crucial to solve multi-agent collaborative (MAC) problems. While centralized reinforcement learning methods can optimally solve small MAC instances, they…
Forward Modeling for Partial Observation Strategy Games - A StarCraft Defogger
Gabriel Synnaeve, Zeming Lin, Jonas Gehring +5
We formulate the problem of defogging as state estimation and future state prediction from previous, partial observations in the context of real-time strategy games. We propose to…
Multi-Organ Cancer Classification and Survival Analysis
Stefan Bauer, Nicolas Carion, Peter Schüffler +3
Accurate and robust cell nuclei classification is the cornerstone for a wider range of tasks in digital and Computational Pathology. However, most machine learning systems require…