8 papers
MemLearner: Learning to Query Context memory for Video World Models
Jiwen Yu, Jianxiong Gao, Jianhong Bai +7
Video World Models are interactive video generation models that predict future world states based on user actions and history video frames. A critical challenge in video world mode…
CineScene: Implicit 3D as Effective Scene Representation for Cinematic Video Generation
Kaiyi Huang, Yukun Huang, Yu Li +8
Cinematic video production requires control over scene-subject composition and camera movement, but live-action shooting remains costly due to the need for constructing physical se…
SemanticGen: Video Generation in Semantic Space
Jianhong Bai, Xiaoshi Wu, Xintao Wang +9
State-of-the-art video generative models typically learn the distribution of video latents in the VAE space and map them to pixels using a VAE decoder. While this approach can gene…
RelightMaster: Precise Video Relighting with Multi-plane Light Images
Weikang Bian, Xiaoyu Shi, Zhaoyang Huang +6
Recent advances in diffusion models enable high-quality video generation and editing, but precise relighting with consistent video contents, which is critical for shaping scene atm…
AdaViewPlanner: Adapting Video Diffusion Models for Viewpoint Planning in 4D Scenes
Yu Li, Menghan Xia, Gongye Liu +7
Recent Text-to-Video (T2V) models have demonstrated powerful capability in visual simulation of real-world geometry and physical laws, indicating its potential as implicit world mo…
CamCloneMaster: Enabling Reference-based Camera Control for Video Generation
Yawen Luo, Jianhong Bai, Xiaoyu Shi +6
Camera control is crucial for generating expressive and cinematic videos. Existing methods rely on explicit sequences of camera parameters as control conditions, which can be cumbe…
Context as Memory: Scene-Consistent Interactive Long Video Generation with Memory Retrieval
Jiwen Yu, Jianhong Bai, Yiran Qin +5
Recent advances in interactive video generation have shown promising results, yet existing approaches struggle with scene-consistent memory capabilities in long video generation du…
Learning Video Generation for Robotic Manipulation with Collaborative Trajectory Control
Xiao Fu, Xintao Wang, Xian Liu +5
Recent advances in video diffusion models shows promise for generating robotic decision-making data, with trajectory conditions further enabling fine-grained control. However, exis…