11 citations · 27 across the 32 of their papers we have counts for
34 papers
Interactive World Simulation with Cosmos for Physical AI
NVIDIA, :, Anis Ahmad +87
We present a shared framework that adapts Cosmos world foundation models for real-time interactive world simulation in Physical AI. By streaming visual observations in response to…
Intrinsic Distribution Matching
Deyuan Liu, Yansen Han, Kaiwen Zheng +7
One-step and few-step text-to-image generation must accurately render the details specified in the prompt despite limited opportunities for refinement. Distribution matching distil…
TurboT2VA: Fast Large-Scale Text-to-Video-Audio Generation via Score-Regularized Consistency Distillation
Xiaoda Yang, Yuxiang Liu, Kaiwen Zheng +12
Joint text-to-video-audio generation produces synchronized visual and acoustic content, but the long sampling trajectories and heterogeneous multimodal computation of large models…
Vidu S1: A Real-Time Interactive Video Generation Model
Jintao Zhang, Kai Jiang, Jintao Chen +24
We introduce Vidu S1, a real-time interactive video generation model supporting voice control of digital characters. Users can control video generation content at any moment throug…
Causal-rCM: A Unified Teacher-Forcing and Self-Forcing Open Recipe for Autoregressive Diffusion Distillation in Streaming Video Generation and Interactive World Models
Kaiwen Zheng, Guande He, Min Zhao +7
Autoregressive video diffusion with causal diffusion transformers has emerged as a major paradigm for real-time streaming video generation and action-conditioned interactive world…
GuidedBridge: Training-freely Improving Bridge Models with Prior Guidance
Zehua Chen, Yucheng Yang, Binjie Yuan +3
Guidance methods, such as classifier-free guidance (CFG) and auto-guidance (AG), have advanced noise-to-data generation in diffusion models. Recently, bridge models have introduced…
Causal Forcing++: Scalable Few-Step Autoregressive Diffusion Distillation for Real-Time Interactive Video Generation
Min Zhao, Hongzhou Zhu, Kaiwen Zheng +6
Real-time interactive video generation requires low-latency, streaming, and controllable rollout. Existing autoregressive (AR) diffusion distillation methods have achieved strong r…
minWM: A Full-Stack Open-Source Framework for Real-Time Interactive Video World Models
Min Zhao, Hongzhou Zhu, Bokai Yan +13
Recent video diffusion foundation models have achieved remarkable progress in high-quality video generation, yet turning them into real-time interactive video world models remains…
SLA2: Sparse-Linear Attention with Learnable Routing and Quantization-aware Training
Jintao Zhang, Haoxu Wang, Kai Jiang +6
Sparse-Linear Attention (SLA) combines sparse and linear attention to accelerate diffusion models and has shown strong performance in video generation. However, (i) SLA relies on a…
Data-regularized Reinforcement Learning for Diffusion Models at Scale
Haotian Ye, Kaiwen Zheng, Jiashu Xu +15
Aligning generative diffusion models with human preferences via reinforcement learning (RL) is critical yet challenging. Most existing algorithms are often vulnerable to reward hac…
Vidarc: Embodied Video Diffusion Model for Closed-loop Control
Yao Feng, Chendong Xiang, Xinyi Mao +7
Robotic arm manipulation in data-scarce settings is a highly challenging task due to the complex embodiment dynamics and diverse contexts. Recent video-based approaches have shown…
TurboDiffusion: Accelerating Video Diffusion Models by 100-200 Times
Jintao Zhang, Kaiwen Zheng, Kai Jiang +5
We introduce TurboDiffusion, a video generation acceleration framework that can speed up end-to-end diffusion generation by 100-200x while maintaining video quality. TurboDiffusion…
World Simulation with Video Foundation Models for Physical AI
NVIDIA, :, Arslan Ali +87
We introduce [Cosmos-Predict2.5], the latest generation of the Cosmos World Foundation Models for Physical AI. Built on a flow-based architecture, [Cosmos-Predict2.5] unifies Text2…
Large Scale Diffusion Distillation via Score-Regularized Continuous-Time Consistency
Kaiwen Zheng, Yuji Wang, Qianli Ma +7
Although continuous-time consistency models (e.g., sCM, MeanFlow) are theoretically principled and empirically powerful for fast academic-scale diffusion, its applicability to larg…
VoiceBridge: General Speech Restoration with One-step Latent Bridge Models
Chi Zhang, Kaiwen Zheng, Zehua Chen +1
Bridge models have been investigated in speech enhancement but are mostly single-task, with constrained general speech restoration (GSR) capability. In this work, we propose VoiceB…
SLA: Beyond Sparsity in Diffusion Transformers via Fine-Tunable Sparse-Linear Attention
Jintao Zhang, Haoxu Wang, Kai Jiang +10
In Diffusion Transformer (DiT) models, particularly for video generation, attention latency is a major bottleneck due to the long sequence length and the quadratic complexity. We f…
AudioMoG: Guiding Audio Generation with Mixture-of-Guidance
Junyou Wang, Zehua Chen, Binjie Yuan +4
The design of diffusion-based audio generation systems has been investigated from diverse perspectives, such as data space, network architecture, and conditioning techniques, while…
DiffusionNFT: Online Diffusion Reinforcement with Forward Process
Kaiwen Zheng, Huayu Chen, Haotian Ye +7
Online reinforcement learning (RL) has been central to post-training language models, but its extension to diffusion models remains challenging due to intractable likelihoods. Rece…
NFT: Bridging Supervised Learning and Reinforcement Learning in Math Reasoning
Huayu Chen, Kaiwen Zheng, Qinsheng Zhang +8
Reinforcement Learning (RL) has played a central role in the recent surge of LLMs' math abilities by enabling self-improvement through binary verifier signals. In contrast, Supervi…
Direct Discriminative Optimization: Your Likelihood-Based Visual Generative Model is Secretly a GAN Discriminator
Kaiwen Zheng, Yongxin Chen, Huayu Chen +4
While likelihood-based generative models, particularly diffusion and autoregressive models, have achieved remarkable fidelity in visual generation, the maximum likelihood estimatio…
Elucidating the Preconditioning in Consistency Distillation
Kaiwen Zheng, Guande He, Jianfei Chen +2
Consistency distillation is a prevalent way for accelerating diffusion models adopted in consistency (trajectory) models, in which a student model is trained to traverse backward o…
Visual Generation Without Guidance
Huayu Chen, Kai Jiang, Kaiwen Zheng +3
Classifier-Free Guidance (CFG) has been a default technique in various visual generative models, yet it requires inference from both conditional and unconditional models during sam…
Consistency Diffusion Bridge Models
Guande He, Kaiwen Zheng, Jianfei Chen +2
Diffusion models (DMs) have become the dominant paradigm of generative modeling in a variety of domains by learning stochastic processes from noise to data. Recently, diffusion den…
Masked Diffusion Models are Secretly Time-Agnostic Masked Models and Exploit Inaccurate Categorical Sampling
Kaiwen Zheng, Yongxin Chen, Hanzi Mao +3
Masked diffusion models (MDMs) have emerged as a popular research topic for generative modeling of discrete data, thanks to their superior performance over other discrete diffusion…