55 citations · 283 across the 50 of their papers we have counts for
56 papers
How to Guide Your Language Flow
Rohit Dilip, Tianrong Chen, Yuyang Wang +3
We introduce a new method to guide flow matching models. Our approach, which we call probe guidance, uses the frozen internal states of an existing diffusion model to construct a g…
SimpleDesign: A Joint Model for Protein Sequence and Structure Codesign
Jiarui Lu, Yuyang Wang, Yizhe Zhang +4
Proteins are fundamental to biological processes, with their function determined by the complex interplay between the amino acid sequence and the three-dimensional structure. Devel…
Normalizing Trajectory Models
Jiatao Gu, Tianrong Chen, Ying Shen +3
Diffusion-based models decompose sampling into many small Gaussian denoising steps -- an assumption that breaks down when generation is compressed to a few coarse transitions. Exis…
Adapting Self-Supervised Representations as a Latent Space for Efficient Generation
Ming Gui, Johannes Schusterbauer, Timy Phan +4
We introduce Representation Tokenizer (RepTok), a generative modeling framework that represents an image using a single continuous latent token obtained from self-supervised vision…
Flexible Language Modeling in Continuous Space with Transformer-based Autoregressive Flows
Ruixiang Zhang, Shuangfei Zhai, Jiatao Gu +6
Autoregressive models have driven remarkable progress in language modeling. Their foundational reliance on discrete tokens, unidirectional context, and single-pass decoding, while…
STARFlow: Scaling Latent Normalizing Flows for High-resolution Image Synthesis
Jiatao Gu, Tianrong Chen, David Berthelot +7
We present STARFlow, a scalable generative model based on normalizing flows that achieves strong performance in high-resolution image synthesis. The core of STARFlow is Transformer…
TADA: Improved Diffusion Sampling with Training-free Augmented Dynamics
Tianrong Chen, Huangjie Zheng, David Berthelot +3
Diffusion models have demonstrated exceptional capabilities in generating high-fidelity images but typically suffer from inefficient sampling. Many solver designs and noise schedul…
Target Concrete Score Matching: A Holistic Framework for Discrete Diffusion
Ruixiang Zhang, Shuangfei Zhai, Yizhe Zhang +4
Discrete diffusion is a promising framework for modeling and generating discrete data. In this work, we present Target Concrete Score Matching (TCSM), a novel and versatile objecti…
3D Shape Tokenization via Latent Flow Matching
Jen-Hao Rick Chang, Yuyang Wang, Miguel Angel Bautista Martin +4
We introduce a latent 3D representation that models 3D surfaces as probability density functions in 3D, i.e., p(x,y,z), with flow-matching. Our representation is specifically desig…
Normalizing Flows are Capable Generative Models
Shuangfei Zhai, Ruixiang Zhang, Preetum Nakkiran +7
Normalizing Flows (NFs) are likelihood-based models for continuous inputs. They have demonstrated promising results on both density estimation and generative modeling tasks, but ha…
INRFlow: Flow Matching for INRs in Ambient Space
Yuyang Wang, Anurag Ranjan, Josh Susskind +1
Flow matching models have emerged as a powerful method for generative modeling on domains like images or videos, and even on irregular or unstructured data like 3D point clouds or…
Multimodal Autoregressive Pre-training of Large Vision Encoders
Enrico Fini, Mustafa Shukor, Xiujun Li +13
We introduce a novel method for pre-training of large-scale vision encoders. Building on recent advancements in autoregressive pre-training of vision models, we extend this framewo…
TypeScore: A Text Fidelity Metric for Text-to-Image Generative Models
Georgia Gabriela Sampaio, Ruixiang Zhang, Shuangfei Zhai +4
Evaluating text-to-image generative models remains a challenge, despite the remarkable progress being made in their overall performances. While existing metrics like CLIPScore work…
DART: Denoising Autoregressive Transformer for Scalable Text-to-Image Generation
Jiatao Gu, Yuyang Wang, Yizhe Zhang +5
Diffusion models have become the dominant approach for visual generation. They are trained by denoising a Markovian process which gradually adds noise to the input. We argue that t…
Improving GFlowNets for Text-to-Image Diffusion Alignment
Dinghuai Zhang, Yizhe Zhang, Jiatao Gu +4
Diffusion models have become the de-facto approach for generating visual data, which are trained to match the distribution of the training dataset. In addition, we also want to con…
Kaleido Diffusion: Improving Conditional Diffusion Models with Autoregressive Latent Modeling
Jiatao Gu, Ying Shen, Shuangfei Zhai +3
Diffusion models have emerged as a powerful tool for generating high-quality images from textual descriptions. Despite their successes, these models often exhibit limited diversity…
CTRLorALTer: Conditional LoRAdapter for Efficient 0-Shot Control & Altering of T2I Models
Nick Stracke, Stefan Andreas Baumann, Joshua M. Susskind +2
Text-to-image generative models have become a prominent and powerful tool that excels at generating high-resolution realistic images. However, guiding the generative process of the…
Many-to-many Image Generation with Auto-regressive Diffusion Models
Ying Shen, Yizhe Zhang, Shuangfei Zhai +3
Recent advancements in image generation have made significant progress, yet existing models present limitations in perceiving and generating an arbitrary number of interrelated ima…
How Far Are We from Intelligent Visual Deductive Reasoning?
Yizhe Zhang, He Bai, Ruixiang Zhang +4
Vision-Language Models (VLMs) have recently demonstrated incredible strides on diverse vision language tasks. We dig into vision-based deductive reasoning, a more sophisticated but…
Scalable Pre-training of Large Autoregressive Image Models
Alaaeldin El-Nouby, Michal Klein, Shuangfei Zhai +5
This paper introduces AIM, a collection of vision models pre-trained with an autoregressive objective. These models are inspired by their textual counterparts, i.e., Large Language…
Swallowing the Bitter Pill: Simplified Scalable Conformer Generation
Yuyang Wang, Ahmed A. Elhag, Navdeep Jaitly +2
We present a novel way to predict molecular conformers through a simple formulation that sidesteps many of the heuristics of prior works and achieves state of the art results by us…
Pseudo-Generalized Dynamic View Synthesis from a Video
Xiaoming Zhao, Alex Colburn, Fangchang Ma +3
Rendering scenes observed in a monocular video from novel viewpoints is a challenging problem. For static scenes the community has studied both scene-specific optimization techniqu…
Generative Modeling with Phase Stochastic Bridges
Tianrong Chen, Jiatao Gu, Laurent Dinh +3
Diffusion models (DMs) represent state-of-the-art generative models for continuous inputs. DMs work by constructing a Stochastic Differential Equation (SDE) in the input space (ie,…
Value function estimation using conditional diffusion models for control
Bogdan Mazoure, Walter Talbott, Miguel Angel Bautista +3
A fairly reliable trend in deep reinforcement learning is that the performance scales with the number of parameters, provided a complimentary scaling in amount of training data. As…