8 papers · 1 filter
Making MLLM Latents Matter: Factor-Aware Predictive Learning for Cinematic Video Generation
Songlin Yang, Zhe Wang, Xiaotong Zhao +8
Cinematic video generation requires both temporal inference from limited inputs (i.e., sparse rather than frame-by-frame guidance) and factor-aware control of character, scene, act…
Provenance-Aware Belief Updating in Human-AI Feedback Loops
Tianyi Huang, Eric Liu, Jay Chong +2
In repeated human-AI interaction, a single observation can be quoted, summarized, and returned to a model as multiple reports, making repetition look like independent corroboration…
Adaptive Reward Routing: Dynamic Multi-Reward Optimization for Joint Audio-Video Diffusion via Forward-Process RL
Songlin Yang, Xiaotong Zhao, Jiacheng Zhang +5
Multi-reward guided reinforcement learning (i.e., RL) offers a promising way to improve joint audio-video diffusion models along several complementary objectives, including modalit…
OmniVBench: A Benchmark and Large-Scale Dataset for Omni Reference-to-Video Generation
Wenxue Li, Peiyan Guan, Haoyang Jiang +15
Omni reference-to-video (R2V) generation is emerging as a general paradigm that enables flexible video customization from diverse and compositional visual references. However, exis…
Temporally Grounded Compositional Camera Motion Understanding via Geometric Knowledge Distillation
Dazhao Du, Shiyan Du, Jian Liu +8
Understanding camera motion is fundamental to video perception, with applications in spatial intelligence and controllable video generation. Multimodal large language models (MLLMs…
Learning Spatiotemporal Sensitivity in Video LLMs via Counterfactual Reinforcement Learning
Dazhao Du, Jian Liu, Jialong Qin +7
Video large language models (Video LLMs) can achieve strong video-QA accuracy without reliably tracking spatiotemporal dynamics. A model may answer a motion question from static cu…
MLLMs Know When Before Speaking: Revealing and Recovering Temporal Grounding via Attention Cues
Dazhao Du, Liao Duan, Jian Liu +5
Video temporal grounding (VTG), which localizes the start and end times of a queried event in an untrimmed video, is a key test of whether multimodal large language models (MLLMs)…
SimWorld: An Open-ended Realistic Simulator for Autonomous Agents in Physical and Social Worlds
Jiawei Ren, Yan Zhuang, Xiaokang Ye +20
While LLM/VLM-powered AI agents have advanced rapidly in math, coding, and computer use, their applications in complex physical and social environments remain challenging. Building…