6 papers · 1 filter
Structure From Tracking: Distilling Structure-Preserving Motion for Video Generation
Yang Fei, George Stoica, Jingyuan Liu +4
Reality is a dance between rigid constraints and deformable structures. For video models, that means generating motion that preserves fidelity as well as structure. Despite progres…
VideoMiner: Iteratively Grounding Key Frames of Hour-Long Videos via Tree-based Group Relative Policy Optimization
Xinye Cao, Hongcan Guo, Jiawen Qian +6
Understanding hour-long videos with multi-modal large language models (MM-LLMs) enriches the landscape of human-centered AI applications. However, for end-to-end video understandin…
How Animals Dance (When You're Not Looking)
Xiaojuan Wang, Aleksander Holynski, Brian Curless +2
We present a framework for generating music-synchronized, choreography aware animal dance videos. Our framework introduces choreography patterns -- structured sequences of motion b…
Generative Inbetweening: Adapting Image-to-Video Models for Keyframe Interpolation
Xiaojuan Wang, Boyang Zhou, Brian Curless +3
We present a method for generating video sequences with coherent motion between a pair of input key frames. We adapt a pretrained large-scale image-to-video diffusion model (origin…
Jump Cut Smoothing for Talking Heads
Xiaojuan Wang, Taesung Park, Yang Zhou +2
A jump cut offers an abrupt, sometimes unwanted change in the viewing experience. We present a novel framework for smoothing these jump cuts, in the context of talking head videos.…
Generative Powers of Ten
Xiaojuan Wang, Janne Kontkanen, Brian Curless +6
We present a method that uses a text-to-image model to generate consistent content across multiple image scales, enabling extreme semantic zooms into a scene, e.g., ranging from a…