1 citations · 3 across the 8 of their papers we have counts for
8 papers
Spatio-Temporal Audio Language Modeling for Dynamic Sound Sources
Oh Hyun-Bin, Kazuki Shimada, Yuhta Takida +6
Sound events have semantic identities, locations, and trajectories, yet existing audio-language models typically encode clips as global event content. Conversely, sound event local…
FacEDiT: Unified Talking Face Editing and Generation via Facial Motion Infilling
Kim Sung-Bin, Joohyun Chang, David Harwath +1
Talking face editing and face generation have often been studied as distinct problems. In this work, we propose viewing both not as separate tasks but as subtasks of a unifying for…
AlignDiT: Multimodal Aligned Diffusion Transformer for Synchronized Speech Generation
Jeongsoo Choi, Ji-Hoon Kim, Kim Sung-Bin +2
In this paper, we address the task of multimodal-to-speech generation, which aims to synthesize high-quality speech from multiple input modalities: text, video, and reference audio…
MultiTalk: Enhancing 3D Talking Head Generation Across Languages with Multilingual Video Dataset
Kim Sung-Bin, Lee Chae-Yeon, Gihun Son +4
Recent studies in speech-driven 3D talking head generation have achieved convincing results in verbal articulations. However, generating accurate lip-syncs degrades when applied to…
SMILE: Multimodal Dataset for Understanding Laughter in Video with Language Models
Lee Hyun, Kim Sung-Bin, Seungju Han +2
Despite the recent advances of the artificial intelligence, building social intelligence remains a challenge. Among social signals, laughter is one of the distinctive expressions t…
LaughTalk: Expressive 3D Talking Head Generation with Laughter
Kim Sung-Bin, Lee Hyun, Da Hye Hong +3
Laughter is a unique expression, essential to affirmative social interactions of humans. Although current 3D talking head generation methods produce convincing verbal articulations…
A Large-Scale 3D Face Mesh Video Dataset via Neural Re-parameterized Optimization
Kim Youwang, Lee Hyun, Kim Sung-Bin +3
We propose NeuFace, a 3D face mesh pseudo annotation method on videos via neural re-parameterized optimization. Despite the huge progress in 3D face reconstruction methods, generat…
Prefix tuning for automated audio captioning
Minkyu Kim, Kim Sung-Bin, Tae-Hyun Oh
Audio captioning aims to generate text descriptions from environmental sounds. One challenge of audio captioning is the difficulty of the generalization due to the lack of audio-te…