9 papers · 1 filter
JAM-Flow: Joint Audio-Motion Synthesis with Flow Matching
Mingi Kwon, Joonghyuk Shin, Jaeseok Jung +2
The intrinsic link between facial motion and speech is often overlooked in generative modeling, where talking head synthesis and text-to-speech (TTS) are typically addressed as sep…
FlowBlending: Stage-Aware Multi-Model Sampling for Fast and High-Fidelity Video Generation
Jibin Song, Mingi Kwon, Jaeseok Jeong +1
In this work, we show that the impact of model capacity varies across timesteps: it is crucial for the early and late stages but largely negligible during the intermediate stage. A…
Balanced conic rectified flow
Shin Seong Kim, Mingi Kwon, Jaeseok Jeong +1
Rectified flow is a generative model that learns smooth transport mappings between two distributions through an ordinary differential equation (ODE). Unlike diffusion-based generat…
Syncphony: Synchronized Audio-to-Video Generation with Diffusion Transformers
Jibin Song, Mingi Kwon, Jaeseok Jeong +1
Text-to-video and image-to-video generation have made rapid progress in visual quality, but they remain limited in controlling the precise timing of motion. In contrast, audio prov…
TCFG: Tangential Damping Classifier-free Guidance
Mingi Kwon, Shin seong Kim, Jaeseok Jeong. Yi Ting Hsiao +1
Diffusion models have achieved remarkable success in text-to-image synthesis, largely attributed to the use of classifier-free guidance (CFG), which enables high-quality, condition…
HARIVO: Harnessing Text-to-Image Models for Video Generation
Mingi Kwon, Seoung Wug Oh, Yang Zhou +6
We present a method to create diffusion-based video models from pretrained Text-to-Image (T2I) models. Recently, AnimateDiff proposed freezing the T2I model while only training tem…