249 citations · 625 across the 53 of their papers we have counts for
56 papers
Normalizing Trajectory Models
Jiatao Gu, Tianrong Chen, Ying Shen +3
Diffusion-based models decompose sampling into many small Gaussian denoising steps -- an assumption that breaks down when generation is compressed to a few coarse transitions. Exis…
STARFlow2: Bridging Language Models and Normalizing Flows for Unified Multimodal Generation
Ying Shen, Tianrong Chen, Yuan Gao +6
Deep generative models have advanced rapidly across text and vision, motivating unified multimodal systems that can understand, reason over, and generate interleaved text-image seq…
Normalizing Flows with Iterative Denoising
Tianrong Chen, Jiatao Gu, David Berthelot +2
Normalizing Flows (NFs) are a classical family of likelihood-based methods that have received revived attention. Recent efforts such as TARFlow have shown that NFs are capable of a…
Exclusive Self Attention
Shuangfei Zhai
We introduce exclusive self attention (XSA), a simple modification of self attention (SA) that improves Transformer's sequence modeling performance. The key idea is to constrain at…
The Coupling Within: Flow Matching via Distilled Normalizing Flows
David Berthelot, Tianrong Chen, Jiatao Gu +6
Flow models have rapidly become the go-to method for training and deploying large-scale generators, owing their success to inference-time flexibility via adjustable integration ste…
STARFlow-V: End-to-End Video Generative Modeling with Normalizing Flows
Jiatao Gu, Ying Shen, Tianrong Chen +6
Normalizing flows (NFs) are end-to-end likelihood-based generative models for continuous data, and have recently regained attention with encouraging progress on image generation. Y…
Flexible Language Modeling in Continuous Space with Transformer-based Autoregressive Flows
Ruixiang Zhang, Shuangfei Zhai, Jiatao Gu +6
Autoregressive models have driven remarkable progress in language modeling. Their foundational reliance on discrete tokens, unidirectional context, and single-pass decoding, while…
STARFlow: Scaling Latent Normalizing Flows for High-resolution Image Synthesis
Jiatao Gu, Tianrong Chen, David Berthelot +7
We present STARFlow, a scalable generative model based on normalizing flows that achieves strong performance in high-resolution image synthesis. The core of STARFlow is Transformer…
TADA: Improved Diffusion Sampling with Training-free Augmented Dynamics
Tianrong Chen, Huangjie Zheng, David Berthelot +3
Diffusion models have demonstrated exceptional capabilities in generating high-fidelity images but typically suffer from inefficient sampling. Many solver designs and noise schedul…
How PARTs assemble into wholes: Learning the relative composition of images
Melika Ayoughi, Samira Abnar, Chen Huang +10
The composition of objects and their parts, along with object-object positional relationships, provides a rich source of information for representation learning. Hence, spatial-awa…
Target Concrete Score Matching: A Holistic Framework for Discrete Diffusion
Ruixiang Zhang, Shuangfei Zhai, Yizhe Zhang +4
Discrete diffusion is a promising framework for modeling and generating discrete data. In this work, we present Target Concrete Score Matching (TCSM), a novel and versatile objecti…
Composition and Control with Distilled Energy Diffusion Models and Sequential Monte Carlo
James Thornton, Louis Bethune, Ruixiang Zhang +3
Diffusion models may be formulated as a time-indexed sequence of energy-based models, where the score corresponds to the negative gradient of an energy function. As opposed to lear…
World-consistent Video Diffusion with Explicit 3D Modeling
Qihang Zhang, Shuangfei Zhai, Miguel Angel Bautista +4
Recent advancements in diffusion models have set new benchmarks in image and video generation, enabling realistic visual synthesis across single- and multi-frame contexts. However,…
Normalizing Flows are Capable Generative Models
Shuangfei Zhai, Ruixiang Zhang, Preetum Nakkiran +7
Normalizing Flows (NFs) are likelihood-based models for continuous inputs. They have demonstrated promising results on both density estimation and generative modeling tasks, but ha…
TypeScore: A Text Fidelity Metric for Text-to-Image Generative Models
Georgia Gabriela Sampaio, Ruixiang Zhang, Shuangfei Zhai +4
Evaluating text-to-image generative models remains a challenge, despite the remarkable progress being made in their overall performances. While existing metrics like CLIPScore work…
DART: Denoising Autoregressive Transformer for Scalable Text-to-Image Generation
Jiatao Gu, Yuyang Wang, Yizhe Zhang +5
Diffusion models have become the dominant approach for visual generation. They are trained by denoising a Markovian process which gradually adds noise to the input. We argue that t…
Improving GFlowNets for Text-to-Image Diffusion Alignment
Dinghuai Zhang, Yizhe Zhang, Jiatao Gu +4
Diffusion models have become the de-facto approach for generating visual data, which are trained to match the distribution of the training dataset. In addition, we also want to con…
Kaleido Diffusion: Improving Conditional Diffusion Models with Autoregressive Latent Modeling
Jiatao Gu, Ying Shen, Shuangfei Zhai +3
Diffusion models have emerged as a powerful tool for generating high-quality images from textual descriptions. Despite their successes, these models often exhibit limited diversity…
Many-to-many Image Generation with Auto-regressive Diffusion Models
Ying Shen, Yizhe Zhang, Shuangfei Zhai +3
Recent advancements in image generation have made significant progress, yet existing models present limitations in perceiving and generating an arbitrary number of interrelated ima…
How Far Are We from Intelligent Visual Deductive Reasoning?
Yizhe Zhang, He Bai, Ruixiang Zhang +4
Vision-Language Models (VLMs) have recently demonstrated incredible strides on diverse vision language tasks. We dig into vision-based deductive reasoning, a more sophisticated but…
Scalable Pre-training of Large Autoregressive Image Models
Alaaeldin El-Nouby, Michal Klein, Shuangfei Zhai +5
This paper introduces AIM, a collection of vision models pre-trained with an autoregressive objective. These models are inspired by their textual counterparts, i.e., Large Language…
Generative Modeling with Phase Stochastic Bridges
Tianrong Chen, Jiatao Gu, Laurent Dinh +3
Diffusion models (DMs) represent state-of-the-art generative models for continuous inputs. DMs work by constructing a Stochastic Differential Equation (SDE) in the input space (ie,…
BOOT: Data-free Distillation of Denoising Diffusion Models with Bootstrapping
Jiatao Gu, Shuangfei Zhai, Yizhe Zhang +2
Diffusion models have demonstrated excellent potential for generating diverse images. However, their performance often suffers from slow generation due to iterative denoising. Know…
PLANNER: Generating Diversified Paragraph via Latent Language Diffusion Model
Yizhe Zhang, Jiatao Gu, Zhuofeng Wu +3
Autoregressive models for text sometimes generate repetitive and low-quality output because errors accumulate during the steps of generation. This issue is often attributed to expo…