11 papers
Dual Exam: Controlled Evaluation of Multimodal Agents and World Models
Zhefan Rao, Meng Chu, Xuan Zhang +93
World Models (WMs) can support Multimodal Agents (MAs) through simulated experience, while Agent trajectories can inform World Model refinement. Evaluating this interaction require…
Generative World Simulation with DAO and JING
XGEN Team, Guohao Cai, Haoxuan Che +16
We present a Generative World Simulation framework that couples persistent shared world state with continuous first-person audiovisual generation. The framework integrates DAO, a c…
The Past Frames the Future: Memory for Autoregressive Video Generation
Harold Haodong Chen, Rongjin Guo, Disen Lan +22
Advances in generative models have improved video fidelity, enabling long-horizon generation, interactive world modeling, and evolving visual environments. Autoregressive (AR) vide…
From Dense Prediction to Visual Editing: Structured Supervision for Unified Image and Video Creation
Zhefan Rao, Bin Zou, Xuanhua He +5
Unified image and video creation requires a model to follow diverse instructions while preserving identity, geometry, and temporal structure from visual context. However, semantic-…
UNISON: A Unified Sound Generation and Editing Framework via Deep LLM Fusion
Zhaoqing Li, Haoning Xu, Jingran Su +9
We present UNISON, a latent diffusion framework that unifies speech generation, sound generation, and audio editing within a single model. A single model handles text-to-audio, tex…
InsEdit: Towards Instruction-based Visual Editing via Data-Efficient Video Diffusion Models Adaptation
Zhefan Rao, Bin Zou, Haoxuan Che +5
Instruction-based video editing is a natural way to control video content with text, but adapting a video generation model into an editor usually appears data-hungry. At the same t…
Agentic World Modeling: Foundations, Capabilities, Laws, and Beyond
Meng Chu, Xuan Billy Zhang, Kevin Qinghong Lin +47
As AI systems move from generating text to accomplishing goals through sustained interaction, the ability to model environment dynamics becomes a central bottleneck. Agents that ma…
VisionDirector: Vision-Language Guided Closed-Loop Refinement for Generative Image Synthesis
Meng Chu, Senqiao Yang, Haoxuan Che +8
Generative models can now produce photorealistic imagery, yet they still struggle with the long, multi-goal prompts that professional designers issue. To expose this gap and better…
ModelGrow: Continual Text-to-Video Pre-training with Model Expansion and Language Understanding Enhancement
Zhefan Rao, Liya Ji, Yazhou Xing +6
Text-to-video (T2V) generation has gained significant attention recently. However, the costs of training a T2V model from scratch remain persistently high, and there is considerabl…
ENTED: Enhanced Neural Texture Extraction and Distribution for Reference-based Blind Face Restoration
Yuen-Fui Lau, Tianjia Zhang, Zhefan Rao +1
We present ENTED, a new framework for blind face restoration that aims to restore high-quality and realistic portrait images. Our method involves repairing a single degraded input…
LDM-ISP: Enhancing Neural ISP for Low Light with Latent Diffusion Models
Qiang Wen, Zhefan Rao, Yazhou Xing +1
Enhancing a low-light noisy RAW image into a well-exposed and clean sRGB image is a significant challenge for modern digital cameras. Prior approaches have difficulties in recoveri…