1 citations · 2 across the 19 of their papers we have counts for
25 papers
Transforming Image Editors into Video Editors
Feng Wang, Zijie Li, Ceyuan Yang +2
Recent image editing systems have achieved impressive semantic understanding, visual fidelity, and instruction-following ability, while video editing remains substantially more dif…
4DCodeBench: Benchmarking Agents on Inverse Graphics of Dynamic Scenes
Ruihong Shen, Žiga Kovačič, Peter Kulits +6
We introduce 4DCodeBench, a benchmark for 4D inverse graphics through code generation, in which agents reconstruct dynamic scenes from video as executable graphics programs. To acc…
Generative Cinematographer: Composing Camera and Object Motion in 3D
Jiahan Zhang, Chaohao Yang, Namitha Guruprasad +4
Current controllable video generation systems often rely on 2D motion trajectories or sparse drag signals for object motion. These controls are ambiguous because the same 2D trajec…
Training Object Permanence in World Models
Haotian Zhang, Fengyuan Yu, Dezhi Luo +28
Object permanence and solidity are hallmarks of human cognitive priors. Recent studies show that video generation models, a paradigmatic class of current world models, have begun t…
VBVR-Pro: A Scalable and Verifiable Suite for Native Visual Reasoning
Junxiang Xu, Ruisi Wang, Fanyi Pu +49
Native visual reasoning treats visual generation as the medium of reasoning itself: visual states (i.e. images and videos) are not merely inputs to be understood or outputs to be r…
MemoBench: Benchmarking World Modeling in Dynamically Changing Environments
Haoyu Chen, Kaichen Zhou, Hang Hua +11
Video generation models aspire to simulate dynamic environments, and several benchmarks now evaluate memory consistency across frames. However, most assess consistency only while t…
Let RGB Be the Language of Vision
Timing Yang, Jinrui Yang, Xinlong Li +11
This work introduces a unified formulation for vision models, where diverse forms of visual information beyond natural images, such as masks, depth maps, and other structured visua…
Large Language Models are Universal Reasoners for Visual Generation
Sucheng Ren, Chen Chen, Zhenbang Wang +5
Text-to-image generation has advanced rapidly with diffusion models, progressing from CLIP and T5 conditioning to unified systems where a single LLM backbone handles both visual un…
VTok: A Unified Video Tokenizer with Decoupled Spatial-Temporal Latents
Feng Wang, Yichun Shi, Ceyuan Yang +4
This work presents VTok, a unified video tokenization framework that can be used for both generation and understanding tasks. Unlike the leading vision-language systems that tokeni…
CausalSpatial: A Benchmark for Object-Centric Causal Spatial Reasoning
Wenxin Ma, Chenlong Wang, Ruisheng Yuan +6
Humans can look at a static scene and instantly predict what happens next -- will moving this object cause a collision? We call this ability Causal Spatial Reasoning. However, curr…
Captain Safari: A World Engine with Pose-Aligned 3D Memory
Yu-Cheng Chou, Xingrui Wang, Yitong Li +5
World engines aim to synthesize long, 3D-consistent videos that support interactive exploration of a scene under user-controlled camera motion. However, existing systems struggle u…
World-in-World: World Models in a Closed-Loop World
Jiahan Zhang, Muqing Jiang, Nanru Dai +14
Generative world models (WMs) can now simulate worlds with striking visual realism, which naturally raises the question of whether they can endow embodied agents with predictive pe…
XModBench: Benchmarking Cross-Modal Capabilities and Consistency in Omni-Language Models
Xingrui Wang, Jiang Liu, Chao Huang +7
Omni-modal large language models (OLLMs) aim to unify audio, vision, and text understanding within a single framework. While existing benchmarks primarily evaluate general cross-mo…
EvoWorld: Evolving Panoramic World Generation with Explicit 3D Memory
Jiahao Wang, Luoxin Ye, TaiMing Lu +8
Humans possess a remarkable ability to mentally explore and replay 3D environments they have previously experienced. Inspired by this mental process, we present EvoWorld: a world m…
Autoregressive Video Generation beyond Next Frames Prediction
Sucheng Ren, Chen Chen, Zhenbang Wang +5
Autoregressive models for video generation typically operate frame-by-frame, extending next-token prediction from language to video's temporal dimension. We question that unlike wo…
Mixture of Contexts for Long Video Generation
Shengqu Cai, Ceyuan Yang, Lvmin Zhang +10
Long video generation is fundamentally a long context memory problem: models must retain and retrieve salient events across a long range without collapsing or drifting. However, sc…
Captain Cinema: Towards Short Movie Generation
Junfei Xiao, Ceyuan Yang, Lvmin Zhang +7
We present Captain Cinema, a generation framework for short movie generation. Given a detailed textual description of a movie storyline, our approach firstly generates a sequence o…
Grouping First, Attending Smartly: Training-Free Acceleration for Diffusion Transformers
Sucheng Ren, Qihang Yu, Ju He +2
Diffusion-based Transformers have demonstrated impressive generative capabilities, but their high computational costs hinder practical deployment, for example, generating an $8192\…
ReVision: Refining Video Diffusion with Explicit 3D Motion Modeling
Qihao Liu, Ju He, Qihang Yu +2
In recent years, video generation has seen significant advancements. However, challenges still persist in generating complex motions and interactions. To address these challenges,…
Beyond Next-Token: Next-X Prediction for Autoregressive Visual Generation
Sucheng Ren, Qihang Yu, Ju He +3
Autoregressive (AR) modeling, known for its next-token prediction paradigm, underpins state-of-the-art language and visual generative models. Traditionally, a ``token'' is treated…
Dictionary-based Framework for Interpretable and Consistent Object Parsing
Tiezheng Zhang, Qihang Yu, Alan Yuille +1
In this work, we present CoCal, an interpretable and consistent object parsing framework based on dictionary-based mask transformer. Designed around Contrastive Components and Logi…
Spatial457: A Diagnostic Benchmark for 6D Spatial Reasoning of Large Multimodal Models
Xingrui Wang, Wufei Ma, Tiezheng Zhang +3
Although large multimodal models (LMMs) have demonstrated remarkable capabilities in visual scene interpretation and reasoning, their capacity for complex and precise 3-dimensional…
GenEx: Generating an Explorable World
Taiming Lu, Tianmin Shu, Junfei Xiao +8
Understanding, navigating, and exploring the 3D physical real world has long been a central challenge in the development of artificial intelligence. In this work, we take a step to…
Generative World Explorer
Taiming Lu, Tianmin Shu, Alan Yuille +2
Planning with partial observation is a central challenge in embodied AI. A majority of prior works have tackled this challenge by developing agents that physically explore their en…