42 papers
WorldStream: Streaming Interactive 3D Worlds with Hierarchical Planning and Persistent State
Xiaoxiao Ma, Weiyang Jin, Yanwen Ma +11
We present WorldStream, an open-ended 3D world model with three simultaneous capabilities: it extends playable terrain beyond any pre-specified map, acts on open-ended language in…
REEL: Learning to Retrieve, Evaluate, and Evolve Visual Memory for Any Long-Horizon Multi-Shot Video Generation
Hang Xu, Guohui Zhang, Xiaoxiao Ma +7
Memory-based long-video generation aims to combine user control with long-range visual consistency. We organize this process into three functional components: a planner, an adapter…
ReCast: Attribution-Oriented Step Representation Learning for LLM-Based Agent Systems
Weilin Jin, Mingyu Wang, Taiyu Zhu +7
In LLM-based agent systems, failures can originate from early steps whose effects propagate through subsequent interactions, making their origins difficult to identify. To trace su…
SGF+: Decoupling Gradient Flows for Autoregressive Video Generation
Zihan Su, Junhao Zhuang, Yaowei Li +10
Autoregressive video generation requires denoising the current frames while writing their key-value representations as context for future predictions. However, these two roles typi…
JoyAI-Voice 2.0: A Full-Continuous Autoregressive Speech Generation Model with Semantic-Acoustic Joint Representation
Yafeng Chen, Boya Dong, Yankun Huang +17
We present JoyAI-Voice~2.0, an end-to-end anthropomorphic speech generation model built upon a fully continuous, dual-encoder architecture. Raw speech is encoded into continuous la…
WorldLine: Action-Driven Visual Simulation for Robotic Manipulation
Shenghe Zheng, Wenbo Li, Jiyao Zhang +4
Real-world robot learning is constrained by the cost of collecting experience and evaluating candidate behaviors. Video generation models offer a scalable foundation for visual sim…
Where and When to Force: Routed Forcing for Streaming Avatars
Zihan Su, Siwen Lu, Junhao Zhuang +6
Audio-driven streaming avatar generation requires real-time synthesis of speech-synchronized videos with dynamic and diverse motion. Self Forcing uses Distribution Matching Distill…
Unfold The World: Factorize 4D Properties in Reinforcing Spatial Reasoning
Yijun Yang, Shenghe Zheng, Wenbo Li +8
Despite the remarkable prowess of Vision-Language Models (VLMs) in general multimodal tasks, they remain fundamentally ``flat'' when reasoning about the physical world. We argue th…
Building Pretraining Data for World Models: An Unreal Engine-Based Pipeline for Action-Conditioned Video Generation
Haoyu Wang, Songchun Zhang, Haoran Li +3
Action-conditioned video models require large-scale visual data paired with control signals that are temporally aligned with the resulting scene transitions. Such supervision is di…
ZimaBlue: Evolving Generalizable World Action Models through Scalable Video Pre-training
Xionghao Wu, Yijun Yang, Shiyang Zhou +17
Robotic manipulation faces a fundamental scaling challenge: robust generalization demands broad physical experience, yet action-labeled robot trajectories are expensive to collect…
Long-Horizon Audio-Visual Generation for Persistent Stories and Interactive Worlds
Nan Duan, Haoyang Huang, Weiyang Jin +13
Video generation is progressing beyond isolated clips toward long-form narratives and interactive worlds, requiring models to preserve identities, follow user controls, and remain…
EchoWM: Open and Enterable Omnimodal World Models
Songchun Zhang, Yaowei Li, Junhao Zhuang +19
We present EchoWM, an omnimodal world model for enterable generative media that responds to continuous navigation while jointly generating 720p video, environmental sound, music an…
JoyAI-Video-Edit: Real-Time Open-Ended Video Editing with Autoregressive Diffusion
Yicheng Xiao, Wenxun Dai, Xinran Qin +22
Real-time video editing requires low-latency causal generation with bounded computational resources while preserving source fidelity and long-term temporal consistency. We present…
Role-Break in Attention Heads: Understanding and Detecting Hallucinations in VLMs
Mingyu Wang, Weilin Jin, Wenbo Li +5
Despite remarkable progress in vision-language generation, Vision-Language Models (VLMs) remain prone to hallucinations, producing content that is inconsistent with or unsupported…
Self Gradient Forcing: Native Long Video Extrapolation
Junhao Zhuang, Shiyi Zhang, Yuxuan Bian +11
Recent autoregressive video diffusion methods are increasingly built upon Self Forcing, where the student is trained on histories produced by its own rollout rather than ground-tru…
Oxygen-TryOn: Fashion-Native Foundation Model for Any-item Virtual Try-On
Yong Liu, Xiaolong Fu, Zihang Xu +10
We present Oxygen-TryOn, a unified foundation model for any-item virtual try-on. Rather than repurposing a general-purpose image editor, Oxygen-TryOn is fashion-native, built for t…
JoyAI-Sim: A Simulation-Enabled Interconversion Toolchain for the Embodied Data Pyramid
Peidong Liu, Yongce Liu, Songyan Guo +34
Generalist robot policies require trustworthy evaluation and robot-usable training data, but both are difficult to scale with physical robots alone. Real-robot trials and demonstra…
Perceptual Flow Matching for Few-Step Generative Modeling
Chuyang Zhao, Yifei Song, Hongfa Wang +7
We propose Perceptual Flow Matching (PFM), a simple yet effective framework for few-step generation in flow-matching models. Rather than performing velocity regression in the conve…
Flash-GRPO: Efficient Alignment for Video Diffusion via One-Step Policy Optimization
Xiaoxuan He, Siming Fu, Zeyue Xue +9
Group Relative Policy Optimization has emerged as essential for aligning video diffusion models with human preferences, but faces a critical computational bottleneck: training a 14…
AnchorEdit: Maintaining Temporal Consistency in Multi-turn Image Editing via Causal Memory
Hang Xu, Xiaoxiao Ma, Guohui Zhang +7
Multi-turn image editing is essential for iterative design, yet current models often struggle with identity drift and error accumulation over successive steps. While existing resea…
Ultra Flash: Scaling Real-Time Streaming Video Generation to High Resolutions
Xin Lu, Luxury, Jie Huang +27
While recent autoregressive video diffusion models achieve remarkable streaming quality, they remain confined to low resolutions (e.g., 480P), leaving efficient, scalable, real-tim…
SpatialWorld: Benchmarking Interactive Spatial Reasoning of Multimodal Agents in Real-World Tasks
Hongcheng Gao, Hailong Qu, Jingyi Tang +18
Spatial reasoning is a foundational capability for multimodal large language models (MLLMs) to perceive and operate within the physical world. However, existing benchmarks predomin…
JoyAI-VL-Interaction: Real-Time Vision-Language Interaction Intelligence
Dingyu Yao, Junhao Zhou, Chenxu Yang +17
Many moments in the real world do not wait for a user to ask. A fire starts on a security monitor, an expression flickers across a video call, or a product a viewer wants flashes b…
Echo-Memory: A Controlled Study of Memory in Action World Models
Wayne King, Zeyue Xue, Yuxuan Bian +13
We present \textbf{Echo-Memory}, a controlled study of memory mechanisms in action-conditioned world models. These models generate multi-segment videos from a first frame, text pro…