24 papers
WorldStream: Streaming Interactive 3D Worlds with Hierarchical Planning and Persistent State
Xiaoxiao Ma, Weiyang Jin, Yanwen Ma +11
We present WorldStream, an open-ended 3D world model with three simultaneous capabilities: it extends playable terrain beyond any pre-specified map, acts on open-ended language in…
SGF+: Decoupling Gradient Flows for Autoregressive Video Generation
Zihan Su, Junhao Zhuang, Yaowei Li +10
Autoregressive video generation requires denoising the current frames while writing their key-value representations as context for future predictions. However, these two roles typi…
Where and When to Force: Routed Forcing for Streaming Avatars
Zihan Su, Siwen Lu, Junhao Zhuang +6
Audio-driven streaming avatar generation requires real-time synthesis of speech-synchronized videos with dynamic and diverse motion. Self Forcing uses Distribution Matching Distill…
Long-Horizon Audio-Visual Generation for Persistent Stories and Interactive Worlds
Nan Duan, Haoyang Huang, Weiyang Jin +13
Video generation is progressing beyond isolated clips toward long-form narratives and interactive worlds, requiring models to preserve identities, follow user controls, and remain…
EchoWM: Open and Enterable Omnimodal World Models
Songchun Zhang, Yaowei Li, Junhao Zhuang +19
We present EchoWM, an omnimodal world model for enterable generative media that responds to continuous navigation while jointly generating 720p video, environmental sound, music an…
Self Gradient Forcing: Native Long Video Extrapolation
Junhao Zhuang, Shiyi Zhang, Yuxuan Bian +11
Recent autoregressive video diffusion methods are increasingly built upon Self Forcing, where the student is trained on histories produced by its own rollout rather than ground-tru…
Flash-GRPO: Efficient Alignment for Video Diffusion via One-Step Policy Optimization
Xiaoxuan He, Siming Fu, Zeyue Xue +9
Group Relative Policy Optimization has emerged as essential for aligning video diffusion models with human preferences, but faces a critical computational bottleneck: training a 14…
Ultra Flash: Scaling Real-Time Streaming Video Generation to High Resolutions
Xin Lu, Luxury, Jie Huang +27
While recent autoregressive video diffusion models achieve remarkable streaming quality, they remain confined to low resolutions (e.g., 480P), leaving efficient, scalable, real-tim…
HPSv3++: Scaling Reward Models Across the Full Spectrum of Diffusion Model Capabilities
Yijun Liu, Jie Huang, Zeyue Xue +5
Reward models guide text-to-image (T2I) systems toward outputs aligned with human preferences. However, typical reward models such as HPSv3 are trained on pre-annotated data from e…
Echo-Memory: A Controlled Study of Memory in Action World Models
Wayne King, Zeyue Xue, Yuxuan Bian +13
We present \textbf{Echo-Memory}, a controlled study of memory mechanisms in action-conditioned world models. These models generate multi-segment videos from a first frame, text pro…
Echo-Infinity: Learning Evolving Memory for Real-Time Infinite Video Generation
Yuxuan Bian, Zeyue Xue, Songchun Zhang +9
We present Echo Infinity, an autoregressive (AR) framework towards real-time infinite video generation that employs a learnable evolving memory to dynamically filter, abstract, and…
LithoGRPO: Fast Inverse Lithography via GRPO Reinforced Flow Matching
Yao Lai, Xuyuan Xiong, Zeyue Xue +7
In semiconductor manufacturing, lithography projects circuit layouts onto silicon wafers through an optical mask. As circuit features shrink below the wavelength of light, optical…
OmniNFT: Modality-wise Omni Diffusion Reinforcement for Joint Audio-Video Generation
Guohui Zhang, XiaoXiao Ma, Jie Huang +9
Recent advances in joint audio-video generation have been remarkable, yet real-world applications demand strong per-modality fidelity, cross-modal alignment, and fine-grained synch…
A Systematic Post-Train Framework for Video Generation
Zeyue Xue, Siming Fu, Jie Huang +9
While large-scale video diffusion models have demonstrated impressive capabilities in generating high-resolution and semantically rich content, a significant gap remains between th…
RAE-AR: Taming Autoregressive Models with Representation Autoencoders
Hu Yu, Hang Xu, Jie Huang +4
The latent space of generative modeling is long dominated by the VAE encoder. The latents from the pretrained representation encoders (e.g., DINO, SigLIP, MAE) are previously consi…
Astrolabe: Steering Forward-Process Reinforcement Learning for Distilled Autoregressive Video Models
Songchun Zhang, Zeyue Xue, Siming Fu +6
Distilled autoregressive (AR) video models enable efficient streaming generation but frequently misalign with human visual preferences. Existing reinforcement learning (RL) framewo…
OmniForcing: Unleashing Real-time Joint Audio-Visual Generation
Yaofeng Su, Yuming Li, Zeyue Xue +7
Recent joint audio-visual diffusion models achieve remarkable generation quality but suffer from high latency due to their bidirectional attention dependencies, hindering real-time…
AEGPO: Adaptive Entropy-Guided Policy Optimization for Diffusion Models
Yuming Li, Qingyu Li, Chengyu Bai +6
Reinforcement learning from human feedback (RLHF) shows promise for aligning diffusion and flow models, yet policy optimization methods such as GRPO suffer from inefficient and sta…
INT v.s. FP: A Comprehensive Study of Fine-Grained Low-bit Quantization Formats
Mengzhao Chen, Meng Wu, Hui Jin +10
Modern AI hardware, such as Nvidia's Blackwell architecture, is increasingly embracing low-precision floating-point (FP) formats to handle the pervasive activation outliers in Larg…
Open-Vocabulary Spatio-Temporal Scene Graph for Robot Perception and Teleoperation Planning
Yi Wang, Zeyu Xue, Mujie Liu +5
Teleoperation via natural-language reduces operator workload and enhances safety in high-risk or remote settings. However, in dynamic remote scenes, transmission latency during bid…
RewardDance: Reward Scaling in Visual Generation
Jie Wu, Yu Gao, Zilyu Ye +9
Reward Models (RMs) are critical for improving generation models via Reinforcement Learning (RL), yet the RM scaling paradigm in visual generation remains largely unexplored. It pr…
WorldWeaver: Generating Long-Horizon Video Worlds via Rich Perception
Zhiheng Liu, Xueqing Deng, Shoufa Chen +7
Generative video modeling has made significant strides, yet ensuring structural and temporal consistency over long sequences remains a challenge. Current methods predominantly rely…
Scaling Law for Quantization-Aware Training
Mengzhao Chen, Chaoyi Zhang, Jing Liu +8
Large language models (LLMs) demand substantial computational and memory resources, creating deployment challenges. Quantization-aware training (QAT) addresses these challenges by…
DanceGRPO: Unleashing GRPO on Visual Generation
Zeyue Xue, Jie Wu, Yu Gao +8
Recent advances in generative AI have revolutionized visual content creation, yet aligning model outputs with human preferences remains a critical challenge. While Reinforcement Le…