2.3k citations · 7.8k across the 98 of their papers we have counts for
116 papers
Correction Training for Diffusion Models
Donghoon Ahn, Runqian Wang, Alexei A. Efros +2
Iterative sampling in diffusion models can lead to error accumulation due to exposure bias. The denoiser is trained on real images interpolated with noise, but at inference its inp…
Base Models Can Reason By Taking a Cue From Training Data
Sophie L. Wang, Amil Dravid, Rulin Shao +3
In this paper, we study how training data creates associations between the tokens at the start of a base model's response and the reasoning behavior that follows. First, we demonst…
MosaiChunk: Compositing Spatio-Temporal Memory for Autoregressive Video Generation
Yiwen Zhang, Haocheng Xi, Michael Tian-Yue Liu +4
Long-horizon autoregressive video generation is limited by a finite context window. When an object or scene falls out of context, its fine-grained visual details may be lost and di…
Bottom-up Modeling of Repeated Elements via Single Image Analysis-by-Synthesis
Syrine Kalleli, Alexei A. Efros, Mathieu Aubry
We address the problem of discovering repeated elements from a single image. In contrast to existing approaches that depend on large annotated datasets, curated multi-image collect…
Balancing Frequencies and Pixels in Flow Matching
Lucas Degeorge, Paul Couairon, Arijit Ghosh +3
Natural images follow a spectral distribution: most signal energy lies in the low spatial frequencies, while the perceptually important structures such as textures and edge…
The Many Senses of Visual Similarity: A Text-Prompted Image Perceptual Metric
Sheng-Yu Wang, Yotam Nitzan, Aaron Hertzmann +4
Human visual similarity judgments are context-dependent. For example, two images may be similar in shape but distinct in color. Existing perceptual similarity metrics, however, col…
Autoregressive Modeling of Film with Applications in Video Montage
Marcelo Sandoval-Castañeda, Fabian Caba Heilbron, Shiry Ginosar +4
This work introduces FilmGPT, an autoregressive transformer designed to address the challenge of video montage--turning a collection of raw, "unwatchable" footage into coherent cin…
The FID Lottery: Quantifying Hidden Randomness in Generative-Model Evaluation
Nicolas Dufour, Alexei A. Efros, Patrick Pérez
The Frechet Inception Distance (FID) is the de facto arbiter of image generation, yet most papers report just a single number from a single trained model using a single sampling se…
Neuron Populations Exhibit Divergent Selectivity with Scale
Amil Dravid, Yasaman Bahri, Alexei A. Efros +1
We investigate whether neuron populations within neural networks evolve predictably with scale, extending scaling laws beyond macroscopic observables such as loss. To probe this qu…
Back into Plato's Cave: Examining Cross-modal Representational Convergence at Scale
A. Sophia Koepke, Daniil Zverev, Shiry Ginosar +1
The Platonic Representation Hypothesis posits that neural networks trained on different modalities (e.g., text and images) converge toward a shared representation of reality. If tr…
It's Never Too Late: Noise Optimization for Collapse Recovery in Trained Diffusion Models
Anne Harrington, A. Sophia Koepke, Shyamgopal Karthik +2
Contemporary text-to-image models exhibit a surprising degree of mode collapse, as can be seen when sampling several images given the same text prompt. Previous work has attempted…
Fast Data Attribution for Text-to-Image Models
Sheng-Yu Wang, Aaron Hertzmann, Alexei A Efros +2
Data attribution for text-to-image models aims to identify the training images that most significantly influenced a generated output. Existing attribution methods involve considera…
Vision Transformers Don't Need Trained Registers
Nick Jiang, Amil Dravid, Alexei Efros +1
We investigate the mechanism underlying a previously identified phenomenon in Vision Transformers - the emergence of high-norm tokens that lead to noisy attention maps (Darcet et a…
Visual Jenga: Discovering Object Dependencies via Counterfactual Inpainting
Anand Bhattad, Konpat Preechakul, Alexei A. Efros
This paper proposes a novel scene understanding task called Visual Jenga. Drawing inspiration from the game Jenga, the proposed task involves progressively removing objects from a…
GPS as a Control Signal for Image Generation
Chao Feng, Ziyang Chen, Aleksander Holynski +2
We show that the GPS tags contained in photo metadata provide a useful control signal for image generation. We train GPS-to-image models and use them for tasks that require a fine-…
Continuous 3D Perception Model with Persistent State
Qianqian Wang, Yifei Zhang, Aleksander Holynski +2
We present a unified framework capable of solving a broad range of 3D tasks. Our approach features a stateful recurrent model that continuously updates its state representation wit…
Prioritized Generative Replay
Renhao Wang, Kevin Frans, Pieter Abbeel +2
Sample-efficient online reinforcement learning often uses replay buffers to store experience for reuse when updating the value function. However, uniform replay is inefficient, sin…
IT: Idempotent Test-Time Training
Nikita Durasov, Assaf Shocher, Doruk Oner +3
Deep learning models often struggle when deployed in real-world settings due to distribution shifts between training and test data. While existing approaches like domain adaptation…
Evaluating Multiview Object Consistency in Humans and Image Models
Tyler Bonnen, Stephanie Fu, Yutong Bai +5
We introduce a benchmark to directly evaluate the alignment between human observers and vision models on a 3D shape inference task. We leverage an experimental design from the cogn…
Diffusion Models as Data Mining Tools
Ioannis Siglidis, Aleksander Holynski, Alexei A. Efros +2
This paper demonstrates how to use generative models trained for image synthesis as tools for visual data mining. Our insight is that since contemporary generative models learn an…
Rethinking Score Distillation as a Bridge Between Image Distributions
David McAllister, Songwei Ge, Jia-Bin Huang +4
Score distillation sampling (SDS) has proven to be an important tool, enabling the use of large-scale diffusion priors for tasks operating in data-poor domains. Unfortunately, SDS…
Interpreting the Weight Space of Customized Diffusion Models
Amil Dravid, Yossi Gandelsman, Kuan-Chieh Wang +4
We investigate the space of weights spanned by a large collection of customized diffusion models. We populate this space by creating a dataset of over 60,000 models, each of which…
Data Attribution for Text-to-Image Models by Unlearning Synthesized Images
Sheng-Yu Wang, Aaron Hertzmann, Alexei A. Efros +2
The goal of data attribution for text-to-image models is to identify the training images that most influence the generation of a new image. Influence is defined such that, for a gi…
Interpreting the Second-Order Effects of Neurons in CLIP
Yossi Gandelsman, Alexei A. Efros, Jacob Steinhardt
We interpret the function of individual neurons in CLIP by automatically describing them using text. Analyzing the direct effects (i.e. the flow from a neuron through the residual…