6 papers
JAM-Flow: Joint Audio-Motion Synthesis with Flow Matching
Mingi Kwon, Joonghyuk Shin, Jaeseok Jung +2
The intrinsic link between facial motion and speech is often overlooked in generative modeling, where talking head synthesis and text-to-speech (TTS) are typically addressed as sep…
FlowBlending: Stage-Aware Multi-Model Sampling for Fast and High-Fidelity Video Generation
Jibin Song, Mingi Kwon, Jaeseok Jeong +1
In this work, we show that the impact of model capacity varies across timesteps: it is crucial for the early and late stages but largely negligible during the intermediate stage. A…
Balanced conic rectified flow
Shin Seong Kim, Mingi Kwon, Jaeseok Jeong +1
Rectified flow is a generative model that learns smooth transport mappings between two distributions through an ordinary differential equation (ODE). Unlike diffusion-based generat…
Syncphony: Synchronized Audio-to-Video Generation with Diffusion Transformers
Jibin Song, Mingi Kwon, Jaeseok Jeong +1
Text-to-video and image-to-video generation have made rapid progress in visual quality, but they remain limited in controlling the precise timing of motion. In contrast, audio prov…
TTS-CtrlNet: Time varying emotion aligned text-to-speech generation with ControlNet
Jaeseok Jeong, Yuna Lee, Mingi Kwon +1
Recent advances in text-to-speech (TTS) have enabled natural speech synthesis, but fine-grained, time-varying emotion control remains challenging. Existing methods often allow only…
TCFG: Tangential Damping Classifier-free Guidance
Mingi Kwon, Shin seong Kim, Jaeseok Jeong. Yi Ting Hsiao +1
Diffusion models have achieved remarkable success in text-to-image synthesis, largely attributed to the use of classifier-free guidance (CFG), which enables high-quality, condition…