222 citations · 226 across the 19 of their papers we have counts for
23 papers
RenderFormer-V2: Neural Rendering with Heterogeneous Scene Primitives
Chong Zeng, Yue Dong, Pieter Peers +2
We present 'RenderFormer-V2', a unified learned transformer-based neural rendering model, complementary to modern physics-based rendering systems, that can handle diverse light-tra…
Ring Forcing: Towards Precise Long-Term Memory for Autoregressive Video Diffusion
Bowen Xue, Brandon Y. Feng, Chenguo Lin +6
Scaling video generation to long durations reveals a critical bottleneck: current models lack robust long-term memory. This deficiency can be studied along two critical aspects: ob…
VBVR-Pro: A Scalable and Verifiable Suite for Native Visual Reasoning
Junxiang Xu, Ruisi Wang, Fanyi Pu +49
Native visual reasoning treats visual generation as the medium of reasoning itself: visual states (i.e. images and videos) are not merely inputs to be understood or outputs to be r…
EchoWM: Open and Enterable Omnimodal World Models
Songchun Zhang, Yaowei Li, Junhao Zhuang +19
We present EchoWM, an omnimodal world model for enterable generative media that responds to continuous navigation while jointly generating 720p video, environmental sound, music an…
Collascope: Supporting Serendipitous Asset Exploration for Collage-Based Storytelling
Jiayi Zhou, Longji Huang, Lvmin Zhang +5
Collage-based storytelling requires visual elements that support emerging narratives and inspire creative reinterpretation. Existing tools, however, rely largely on keyword- and im…
FlexComposer: Unified Video Compositing from Images to Dynamic Footage with Flexible Trajectory Control
Songchun Zhang, Sitong Guo, Xianghao Kong +4
Generative video compositing, which involves inserting external assets seamlessly into existing video sequences, is essential for content creation and visual effects. However, exis…
Masked Visual Actions for Unified World Modeling
Hadi Alzayer, Wenlong Huang, Haonan Chen +8
Video models absorb rich priors over how the visual world moves, interacts, and responds to contact, making them promising substrates for robotic world modeling. The central challe…
FourTune: Towards Fully 4-Bit Efficient Post-Training for Diffusion Models
Bowen Xue, Zihan Min, Xingyang Li +8
Diffusion models have become a dominant paradigm for high-quality generative modeling, while post-training is essential for adapting them to diverse downstream applications. Howeve…
EvalVerse: Pipeline-Aware and Expert-Calibrated Benchmarking for Professional Cinematic Video Generation
Songlin Yang, Haobin Zhong, Ruilin Zhang +23
The rapid evolution of generative video foundation models has propelled the field toward professional-grade cinematic synthesis. To achieve such demanding quality, the community tr…
UniVidX: A Unified Multimodal Framework for Versatile Video Generation via Diffusion Priors
Houyuan Chen, Hong Li, Xianghao Kong +8
Recent progress has shown that video diffusion models (VDMs) can be repurposed for diverse multimodal graphics tasks. However, existing methods often train separate models for each…
View-oriented Conversation Compiler for Agent Trace Analysis
Lvmin Zhang, Maneesh Agrawala
We observe that an agent trace is a structured document. A coding agent session contains user turns, assistant text, chain of thought blocks, tool calls, tool results, subagent inv…
Mode Seeking meets Mean Seeking for Fast Long Video Generation
Shengqu Cai, Weili Nie, Chao Liu +8
Scaling video generation from seconds to minutes faces a critical bottleneck: while short-video data is abundant and high-fidelity, coherent long-form data is scarce and limited to…
TinyHistory: Lightweight Video History Embeddings via Two-Stage Context Learning
Lvmin Zhang, Shengqu Cai, Muyang Li +6
History context is central to autoregressive video generation, driving consistency and storytelling for both commercial models and personal use cases. For example, personal users,…
StreamDiffusionV2: A Streaming System for Dynamic and Interactive Video Generation
Tianrui Feng, Zhi Li, Shuo Yang +11
Generative models are reshaping the live-streaming industry by redefining how content is created, styled, and delivered. Previous image-based streaming diffusion models have powere…
Taming Flow-based I2V Models for Creative Video Editing
Xianghao Kong, Hansheng Chen, Yuwei Guo +4
Although image editing techniques have advanced significantly, video editing, which aims to manipulate videos according to user intent, remains an emerging challenge. Most existing…
CineVision: An Interactive Pre-visualization Storyboard System for Director-Cinematographer Collaboration
Zheng Wei, Hongtao Wu, Lvmin Zhang +6
Effective communication between directors and cinematographers is fundamental in film production, yet traditional approaches relying on visual references and hand-drawn storyboards…
Mixture of Contexts for Long Video Generation
Shengqu Cai, Ceyuan Yang, Lvmin Zhang +10
Long video generation is fundamentally a long context memory problem: models must retain and retrieve salient events across a long range without collapsing or drifting. However, sc…
Captain Cinema: Towards Short Movie Generation
Junfei Xiao, Ceyuan Yang, Lvmin Zhang +7
We present Captain Cinema, a generation framework for short movie generation. Given a detailed textual description of a movie storyline, our approach firstly generates a sequence o…
Radial Attention: Sparse Attention with Energy Decay for Long Video Generation
Xingyang Li, Muyang Li, Tianle Cai +11
Recent advances in diffusion models have enabled high-quality video generation, but the additional temporal dimension significantly increases computational costs, making training a…
Frame Context Packing and Drift Prevention in Next-Frame-Prediction Video Diffusion Models
Lvmin Zhang, Shengqu Cai, Muyang Li +2
We present a neural network structure, FramePack, to train next-frame (or next-frame-section) prediction models for video generation. FramePack compresses input frame contexts with…
Instance Segmentation of Scene Sketches Using Natural Image Priors
Mia Tang, Yael Vinker, Chuan Yan +2
Sketch segmentation involves grouping pixels within a sketch that belong to the same object or instance. It serves as a valuable tool for sketch editing tasks, such as moving, scal…
Transparent Image Layer Diffusion using Latent Transparency
Lvmin Zhang, Maneesh Agrawala
We present LayerDiffuse, an approach enabling large-scale pretrained latent diffusion models to generate transparent images. The method allows generation of single transparent imag…
Adding Conditional Control to Text-to-Image Diffusion Models
Lvmin Zhang, Anyi Rao, Maneesh Agrawala
We present ControlNet, a neural network architecture to add spatial conditioning controls to large, pretrained text-to-image diffusion models. ControlNet locks the production-ready…