9 papers
AlayaVista: Streaming World Modeling from Panoramic States to Perspective Video
Jiaming Tan, Mingliang Zhai, Zhen Li +3
Interactive video world models must maintain broad scene context under camera motion while producing high-fidelity observations with low latency. Existing approaches face a represe…
AlayaWorld: Interactive Long-Horizon World Modeling - Full Technical Report (v1.1)
AlayaWorld Team, Kaipeng Zhang, Chuanhao Li +15
This report presents an improved version of AlayaWorld. While the backbone architecture, chunk-wise autoregressive generation scheme, and training data remain unchanged from the pr…
AlayaWorld: Interactive Long-Horizon World Modeling -- Full Technical Report
AlayaWorld Team, Kaipeng Zhang, Chuanhao Li +15
Unlike conventional video game development, which relies on labor-intensive pipelines for asset production, animation, physics, and programming, video world models generate interac…
From Pixels to States: Rethinking Interactive World Models as Game Engines
Zhen Li, Zian Meng, Shuwei Shi +4
Building interactive worlds that respond coherently to player actions has long been a shared goal of computer graphics, games, and artificial intelligence. Recent video generative…
ProSoftArena: Benchmarking Hierarchical Capabilities of Multimodal Agents in Professional Software Environments
Jiaxin Ai, Yukang Feng, Fanrui Zhang +8
Multimodal agents are making rapid progress on general computer-use tasks, yet existing benchmarks remain largely confined to browsers and basic desktop applications, falling short…
Multi-Step Reasoning for Embodied Question Answering via Tool Augmentation
Mingliang Zhai, Hansheng Liang, Xiaomeng Fan +6
Embodied Question Answering (EQA) requires agents to explore 3D environments to obtain observations and answer questions related to the scene. Existing methods leverage VLMs to dir…
IA-T2I: Internet-Augmented Text-to-Image Generation
Chuanhao Li, Jianwen Sun, Yukang Feng +3
Current text-to-image (T2I) generation models achieve promising results, but they fail on the scenarios where the knowledge implied in the text prompt is uncertain. For example, a…
Memory-Centric Embodied Question Answering
Mingliang Zhai, Zhi Gao, Yuwei Wu +1
Embodied Question Answering (EQA) requires agents to autonomously explore and comprehend the environment to answer context-dependent questions. Typically, an EQA framework consists…
World knowledge-enhanced Reasoning Using Instruction-guided Interactor in Autonomous Driving
Mingliang Zhai, Cheng Li, Zengyuan Guo +7
The Multi-modal Large Language Models (MLLMs) with extensive world knowledge have revitalized autonomous driving, particularly in reasoning tasks within perceivable regions. Howeve…