14 papers
Making MLLM Latents Matter: Factor-Aware Predictive Learning for Cinematic Video Generation
Songlin Yang, Zhe Wang, Xiaotong Zhao +8
Cinematic video generation requires both temporal inference from limited inputs (i.e., sparse rather than frame-by-frame guidance) and factor-aware control of character, scene, act…
Adaptive Reward Routing: Dynamic Multi-Reward Optimization for Joint Audio-Video Diffusion via Forward-Process RL
Songlin Yang, Xiaotong Zhao, Jiacheng Zhang +5
Multi-reward guided reinforcement learning (i.e., RL) offers a promising way to improve joint audio-video diffusion models along several complementary objectives, including modalit…
RewardVerse: Rubric-Guided Policy Optimization for Video Reward Modeling
Zhenchen Tang, Yang Li, Songlin Yang +6
Reinforcement learning (RL) is vital for optimizing video generation models, with a robust reward model (RM) serving as the cornerstone. However, existing video reward models often…
OmniVBench: A Benchmark and Large-Scale Dataset for Omni Reference-to-Video Generation
Wenxue Li, Peiyan Guan, Haoyang Jiang +15
Omni reference-to-video (R2V) generation is emerging as a general paradigm that enables flexible video customization from diverse and compositional visual references. However, exis…
WorldCycle: Self-Verifiable Reinforcement Learning for Long-Horizon Video World Models
Bohai Gu, Yueyang Yuan, Taiyi Wu +9
Interactive video world models are essential for long-horizon planning and exploration, yet they suffer from compounding errors. Post-training methods such as reinforcement learnin…
WorldCraft: From Camera Navigation to Object Manipulation in Interactive Video World Models
Bohai Gu, Taiyi Wu, Yueyang Yuan +9
Recent video world models enable interactive camera navigation but provide limited control over individual objects. We study persistent object-state control in autoregressive video…
EvalVerse: Pipeline-Aware and Expert-Calibrated Benchmarking for Professional Cinematic Video Generation
Songlin Yang, Haobin Zhong, Ruilin Zhang +23
The rapid evolution of generative video foundation models has propelled the field toward professional-grade cinematic synthesis. To achieve such demanding quality, the community tr…
CogOmniControl: Reasoning-Driven Controllable Video Generation via Creative Intent Cognition
Hongji Yang, Songlian Li, Yucheng Zhou +4
Recent diffusion models achieve strong photorealism and fluency in video generation, yet remain fragile under abstract, sparse or complex conditions, leading to poor performance in…
A Readiness-Driven Runtime for Pipeline-Parallel Training under Runtime Variability
Ruitao Liu, Xinyang Tian, Shuo Chen +4
Pipeline parallelism is a key technique for scaling large-model training, but modern workloads exhibit runtime variability in computation and communication. Existing pipeline syste…
Place-it-R1: Unlocking Environment-aware Reasoning Potential of MLLM for Video Object Insertion
Bohai Gu, Taiyi Wu, Dazhao Du +5
Video object insertion is fundamental to video editing, yet existing diffusion methods often produce visually plausible but physically inconsistent results. We present Place-it-R1,…
Pusa V1.0: Unlocking Temporal Control in Pretrained Video Diffusion Models via Vectorized Timestep Adaptation
Yaofang Liu, Yumeng Ren, Aitor Artola +9
The rapid advancement of video diffusion models has been hindered by fundamental limitations in temporal modeling, particularly the rigid synchronization of frame evolution imposed…
Crab: A Unified Audio-Visual Scene Understanding Model with Explicit Cooperation
Henghui Du, Guangyao Li, Chang Zhou +3
In recent years, numerous tasks have been proposed to encourage model to develop specified capability in understanding audio-visual scene, primarily categorized into temporal local…
3D Dynamic Fluid Assets from Single-View Videos with Generative Gaussian Splatting
Zhiwei Zhao, Alan Zhao, Minchen Li +1
While the generation of 3D content from single-view images has been extensively studied, the creation of physically consistent 3D dynamic scenes from videos remains in its early st…
Dragen3D: Multiview Geometry Consistent 3D Gaussian Generation with Drag-Based Control
Jinbo Yan, Alan Zhao, Yixin Hu
Single-image 3D generation has emerged as a prominent research topic, playing a vital role in virtual reality, 3D modeling, and digital content creation. However, existing methods…