8 papers
Discrete Diffusion for Generative Modeling of Text-Aligned Speech Tokens
Pin-Jui Ku, He Huang, Jean-Marie Lemercier +3
This paper introduces a discrete diffusion model (DDM) framework for text-aligned speech tokenization and reconstruction. By replacing the auto-regressive speech decoder with a dis…
TalkVid: A Large-Scale Diversified Dataset for Audio-Driven Talking Head Synthesis
Shunian Chen, Hejin Huang, Yexin Liu +10
Audio-driven talking head synthesis has achieved remarkable photorealism, yet state-of-the-art (SOTA) models exhibit a critical failure: they lack generalization to the full spectr…
Streaming Sortformer: Speaker Cache-Based Online Speaker Diarization with Arrival-Time Ordering
Ivan Medennikov, Taejin Park, Weiqing Wang +5
This paper presents a streaming extension for the Sortformer speaker diarization framework, whose key property is the arrival-time ordering of output speakers. The proposed approac…
Recent Trends in Distant Conversational Speech Recognition: A Review of CHiME-7 and 8 DASR Challenges
Samuele Cornell, Christoph Boeddeker, Taejin Park +9
The CHiME-7 and 8 distant speech recognition (DASR) challenges focus on multi-channel, generalizable, joint automatic speech recognition (ASR) and diarization of conversational spe…
Speaker Targeting via Self-Speaker Adaptation for Multi-talker ASR
Weiqing Wang, Taejin Park, Ivan Medennikov +6
We propose a self-speaker adaptation method for streaming multi-talker automatic speech recognition (ASR) that eliminates the need for explicit speaker queries. Unlike conventional…
Word Level Timestamp Generation for Automatic Speech Recognition and Translation
Ke Hu, Krishna Puvvada, Elena Rastorgueva +7
We introduce a data-driven approach for enabling word-level timestamp prediction in the Canary model. Accurate timestamp information is crucial for a variety of downstream tasks su…