11 papers
TRACE: Object Motion Editing in Videos with First-Frame Trajectory Guidance
Quynh Phung, Long Mai, Cusuh Ham +3
We study object motion path editing in videos, where the goal is to alter a target object's trajectory while preserving the original scene content. Unlike prior video editing metho…
TokenDial: Continuous Attribute Control for Text-to-Video Generation in Visual Dial Space
Zhixuan Liu, Peter Schaldenbrand, Yijun Li +5
In video diffusion transformers, visual patch tokens maintain explicit correspondence to space and time. We hypothesize that their channel dimension can serve as a semantic control…
V2M-Zero: Zero-Pair Time-Aligned Video-to-Music Generation
Yan-Bo Lin, Jonah Casebeer, Long Mai +3
Generating music that temporally aligns with video events is challenging for existing text-to-music models, which lack fine-grained temporal control. We introduce V2M-ZERO, a video…
DreamLoop: Controllable Cinemagraph Generation from a Single Photograph
Aniruddha Mahapatra, Long Mai, Cusuh Ham +1
Cinemagraphs, which combine static photographs with selective, looping motion, offer unique artistic appeal. Generating them from a single photograph in a controllable manner is pa…
Diffusion Transformer-to-Mamba Distillation for High-Resolution Image Generation
Yuan Yao, Yicong Hong, Difan Liu +3
The quadratic computational complexity of self-attention in diffusion transformers (DiT) introduces substantial computational costs in high-resolution image generation. While the l…
CineVerse: Consistent Keyframe Synthesis for Cinematic Scene Composition
Quynh Phung, Long Mai, Fabian David Caba Heilbron +3
We present CineVerse, a novel framework for the task of cinematic scene composition. Similar to traditional multi-shot generation, our task emphasizes the need for consistency and…
REGEN: Learning Compact Video Embedding with (Re-)Generative Decoder
Yitian Zhang, Long Mai, Aniruddha Mahapatra +5
We present a novel perspective on learning video embedders for generative modeling: rather than requiring an exact reproduction of an input video, an effective embedder should focu…
MotionCanvas: Cinematic Shot Design with Controllable Image-to-Video Generation
Jinbo Xing, Long Mai, Cusuh Ham +5
This paper presents a method that allows users to design cinematic video shots in the context of image-to-video generation. Shot design, a critical aspect of filmmaking, involves m…
Pushing the Boundaries of State Space Models for Image and Video Generation
Yicong Hong, Long Mai, Yuan Yao +1
While Transformers have become the dominant architecture for visual generation, linear attention models, such as the state-space models (SSM), are increasingly recognized for their…
GaussianVideo: Efficient Video Representation via Hierarchical Gaussian Splatting
Andrew Bond, Jui-Hsien Wang, Long Mai +2
Efficient neural representations for dynamic video scenes are critical for applications ranging from video compression to interactive simulations. Yet, existing methods often face…
Progressive Growing of Video Tokenizers for Temporally Compact Latent Spaces
Aniruddha Mahapatra, Long Mai, David Bourgin +2
Video tokenizers are essential for latent video diffusion models, converting raw video data into spatiotemporally compressed latent spaces for efficient training. However, extendin…