12 papers
WorldStream: Streaming Interactive 3D Worlds with Hierarchical Planning and Persistent State
Xiaoxiao Ma, Weiyang Jin, Yanwen Ma +11
We present WorldStream, an open-ended 3D world model with three simultaneous capabilities: it extends playable terrain beyond any pre-specified map, acts on open-ended language in…
Building Pretraining Data for World Models: An Unreal Engine-Based Pipeline for Action-Conditioned Video Generation
Haoyu Wang, Songchun Zhang, Haoran Li +3
Action-conditioned video models require large-scale visual data paired with control signals that are temporally aligned with the resulting scene transitions. Such supervision is di…
Long-Horizon Audio-Visual Generation for Persistent Stories and Interactive Worlds
Nan Duan, Haoyang Huang, Weiyang Jin +13
Video generation is progressing beyond isolated clips toward long-form narratives and interactive worlds, requiring models to preserve identities, follow user controls, and remain…
EchoWM: Open and Enterable Omnimodal World Models
Songchun Zhang, Yaowei Li, Junhao Zhuang +19
We present EchoWM, an omnimodal world model for enterable generative media that responds to continuous navigation while jointly generating 720p video, environmental sound, music an…
Self Gradient Forcing: Native Long Video Extrapolation
Junhao Zhuang, Shiyi Zhang, Yuxuan Bian +11
Recent autoregressive video diffusion methods are increasingly built upon Self Forcing, where the student is trained on histories produced by its own rollout rather than ground-tru…
Ultra Flash: Scaling Real-Time Streaming Video Generation to High Resolutions
Xin Lu, Luxury, Jie Huang +27
While recent autoregressive video diffusion models achieve remarkable streaming quality, they remain confined to low resolutions (e.g., 480P), leaving efficient, scalable, real-tim…
HPSv3++: Scaling Reward Models Across the Full Spectrum of Diffusion Model Capabilities
Yijun Liu, Jie Huang, Zeyue Xue +5
Reward models guide text-to-image (T2I) systems toward outputs aligned with human preferences. However, typical reward models such as HPSv3 are trained on pre-annotated data from e…
Echo-Memory: A Controlled Study of Memory in Action World Models
Wayne King, Zeyue Xue, Yuxuan Bian +13
We present \textbf{Echo-Memory}, a controlled study of memory mechanisms in action-conditioned world models. These models generate multi-segment videos from a first frame, text pro…
IMUG-Bench: Benchmarking Unified Multimodal Models on Interleaved Understanding and Generation
Lingyi Meng, Zecong Tang, Haoran Li +12
In recent years, unified multimodal models (UMMs) have emerged to support both understanding and generation within a single framework. Mastering dynamic, multi-turn interleaved ima…
Echo-Infinity: Learning Evolving Memory for Real-Time Infinite Video Generation
Yuxuan Bian, Zeyue Xue, Songchun Zhang +9
We present Echo Infinity, an autoregressive (AR) framework towards real-time infinite video generation that employs a learnable evolving memory to dynamically filter, abstract, and…
A Systematic Post-Train Framework for Video Generation
Zeyue Xue, Siming Fu, Jie Huang +9
While large-scale video diffusion models have demonstrated impressive capabilities in generating high-resolution and semantically rich content, a significant gap remains between th…
OmniForcing: Unleashing Real-time Joint Audio-Visual Generation
Yaofeng Su, Yuming Li, Zeyue Xue +7
Recent joint audio-visual diffusion models achieve remarkable generation quality but suffer from high latency due to their bidirectional attention dependencies, hindering real-time…