112 citations · 552 across the 46 of their papers we have counts for
46 papers
Flex-Forcing: Towards a Unified Autoregressive and Bidirectional Video Diffusion Model
Xinyin Ma, Julius Berner, Chao Liu +3
Recent progress in large-scale generative models has substantially advanced video generation, yet existing methods remain constrained by a rigid inference paradigm. Bidirectional d…
Mode Seeking meets Mean Seeking for Fast Long Video Generation
Shengqu Cai, Weili Nie, Chao Liu +8
Scaling video generation from seconds to minutes faces a critical bottleneck: while short-video data is abundant and high-fidelity, coherent long-form data is scarce and limited to…
OpenVision 3: A Family of Unified Visual Encoder for Both Understanding and Generation
Letian Zhang, Sucheng Ren, Yanqing Liu +9
This paper presents a family of advanced vision encoder, named OpenVision 3, that learns a single, unified visual representation that can serve both image understanding and image g…
Transition Matching Distillation for Fast Video Generation
Weili Nie, Julius Berner, Nanye Ma +3
Large video diffusion and flow models have achieved remarkable success in high-quality video generation, but their use in real-time interactive applications remains limited due to…
PixelDiT: Pixel Diffusion Transformers for Image Generation
Yongsheng Yu, Wei Xiong, Weili Nie +3
Latent-space modeling has been the standard for Diffusion Transformers (DiTs). However, it relies on a two-stage pipeline where the pretrained autoencoder introduces lossy reconstr…
Exploring Synthesizable Chemical Space with Iterative Pathway Refinements
Seul Lee, Karsten Kreis, Srimukh Prasad Veccham +5
A well-known pitfall of molecular generative models is that they are not guaranteed to generate synthesizable molecules. Existing solutions for this problem often struggle to effec…
One-step Diffusion Models with -Divergence Distribution Matching
Yilun Xu, Weili Nie, Arash Vahdat
Sampling from diffusion models involves a slow iterative process that hinders their practical deployment, especially for interactive applications. To accelerate generation speed, r…
BlobGEN-Vid: Compositional Text-to-Video Generation with Blob Video Representations
Weixi Feng, Chao Liu, Sifei Liu +3
Existing video generation models struggle to follow complex text prompts and synthesize multiple objects, raising the need for additional grounding input for improved controllabili…
A2SB: Audio-to-Audio Schrodinger Bridges
Zhifeng Kong, Kevin J Shih, Weili Nie +6
Real-world audio is often degraded by numerous factors. This work presents an audio restoration model tailored for high-res music at 44.1kHz. Our model, Audio-to-Audio Schrödinger…
GenMol: A Drug Discovery Generalist with Discrete Diffusion
Seul Lee, Karsten Kreis, Srimukh Prasad Veccham +6
Drug discovery is a complex process that involves multiple stages and tasks. However, existing molecular generative models can only tackle some of these tasks. We present Generalis…
Molecule Generation with Fragment Retrieval Augmentation
Seul Lee, Karsten Kreis, Srimukh Prasad Veccham +5
Fragment-based drug discovery, in which molecular fragments are assembled into new molecules with desirable biochemical properties, has achieved great success. However, many fragme…
Warped Diffusion: Solving Video Inverse Problems with Image Diffusion Models
Giannis Daras, Weili Nie, Karsten Kreis +4
Using image models naively for solving inverse video problems often suffers from flickering, texture-sticking, and temporal inconsistency in generated videos. To tackle these probl…
Multi-student Diffusion Distillation for Better One-step Generators
Yanke Song, Jonathan Lorraine, Weili Nie +2
Diffusion models achieve high-quality sample generation at the cost of a lengthy multistep inference procedure. To overcome this, diffusion distillation techniques produce student…
Energy-Based Diffusion Language Models for Text Generation
Minkai Xu, Tomas Geffner, Karsten Kreis +5
Despite remarkable progress in autoregressive language models, alternative generative paradigms beyond left-to-right generation are still being actively explored. Discrete diffusio…
Truncated Consistency Models
Sangyun Lee, Yilun Xu, Tomas Geffner +4
Consistency models have recently been introduced to accelerate sampling from diffusion models by directly predicting the solution (i.e., data) of the probability flow ODE (PF ODE)…
Aligning Target-Aware Molecule Diffusion Models with Exact Energy Optimization
Siyi Gu, Minkai Xu, Alexander Powers +6
Generating ligand molecules for specific protein targets, known as structure-based drug design, is a fundamental problem in therapeutics development and biological discovery. Recen…
CamCo: Camera-Controllable 3D-Consistent Image-to-Video Generation
Dejia Xu, Weili Nie, Chao Liu +4
Recently video diffusion models have emerged as expressive generative tools for high-quality video content creation readily available to general users. However, these models often…
DiffUHaul: A Training-Free Method for Object Dragging in Images
Omri Avrahami, Rinon Gal, Gal Chechik +4
Text-to-image diffusion models have proven effective for solving many image editing tasks. However, the seemingly straightforward task of seamlessly relocating objects within a sce…
Compositional Text-to-Image Generation with Dense Blob Representations
Weili Nie, Sifei Liu, Morteza Mardani +3
Existing text-to-image models struggle to follow complex text prompts, raising the need for extra grounding inputs for better controllability. In this work, we propose to decompose…
Efficient Video Diffusion Models via Content-Frame Motion-Latent Decomposition
Sihyun Yu, Weili Nie, De-An Huang +3
Video diffusion models have recently made great progress in generation quality, but are still limited by the high memory and computational requirements. This is because current vid…
T-Stitch: Accelerating Sampling in Pre-Trained Diffusion Models with Trajectory Stitching
Zizheng Pan, Bohan Zhuang, De-An Huang +5
Sampling from diffusion probabilistic models (DPMs) is often expensive for high-quality image generation and typically requires many steps with a large model. In this paper, we int…
Unsupervised Discovery of Steerable Factors When Graph Deep Generative Models Are Entangled
Shengchao Liu, Chengpeng Wang, Jiarui Lu +5
Deep generative models (DGMs) have been widely developed for graph data. However, much less investigation has been carried out on understanding the latent space of such pretrained…
Augmented Bridge Matching
Valentin De Bortoli, Guan-Horng Liu, Tianrong Chen +2
Flow and bridge matching are a novel class of processes which encompass diffusion models. One of the main aspect of their increased flexibility is that these models can interpolate…
DeepSpeed4Science Initiative: Enabling Large-Scale Scientific Discovery through Sophisticated AI System Technologies
Shuaiwen Leon Song, Bonnie Kruft, Minjia Zhang +89
In the upcoming decade, deep learning may revolutionize the natural sciences, enhancing our capacity to model and predict natural occurrences. This could herald a new era of scient…