19 citations · 53 across the 45 of their papers we have counts for
10 papers · 1 filter
ASSET: Adaptive Speech Segmentation via Reinforcement Learning
Cihan Xiao, Yuan Tseng, Matthew Wiesner +4
Speech large language models (LLMs) are often built using pretrained acoustic encoders as an audio interface to an LLM decoder. However, LLMs scale poorly with input sequence lengt…
RUBATO: Decoupled Clocks Allow Full-Duplex Models to Think as Fast as They Can
Yi-Jen Shih, Puyuan Peng, Haohe Liu +2
Recent Speech Full-Duplex (FDX) models are capable of engaging in naturalistic conversation by listening and speaking at the same time. Building on this foundation, recent architec…
Voices as Handles: Reasoning about Speaker Identity with Frozen Text LLMs
Runqiu Xu, Zhisheng Zheng, David Harwath
Multi-user voice agents must track who said what across dialogue sessions. Text LLMs are attractive backbones for such agents, but transcripts alone do not expose acoustic speaker…
SPADE: A Multilingual Dataset for Speech Partial Deepfake Detection and Localization
Yuan Tseng, Aishwarya Fursule, Andrew Zijun Ma +4
Recent improvements in voice-cloning speech generation systems raise concerns about misuse by malicious actors to impersonate others and spread misinformation. Detecting such tampe…
MP-Bench: Evaluating Voice Agents as a Multiparty Conversation Participant
Yi-Jen Shih, Shih-Yun Shan Kuan, Guan-Ting Lin +7
Conversational voice agents have advanced significantly, offering increasingly natural human-machine interactions through both cascaded and end-to-end architectures. However, while…
Quantifying Consonant Contributions to Word Intelligibility via Acoustic Masking
Eunjung Yeo, Kwanghee Choi, Krupaben Kothadia +4
Consonants contribute unequally to whether a word is understood. Given the limited time available for therapy, ranking consonants by contribution to intelligibility helps prioritiz…
RetroThinker: Enabling Retrospective Thinking in Speech LLMs
Yi-Jen Shih, Puyuan Peng, Abdelrahman Mohamed +1
Speech large language models (SpeechLLMs) offer reduced latency and retain paralinguistic nuances that are typically lost in cascaded automatic speech recognition (ASR) and text-ba…
CtrlSpeech: Coarse-to-Fine Control for Expressive Speech Synthesis
Zhisheng Zheng, Xiaohang Sun, Zhu Liu +5
Recent Text-To-Speech (TTS) systems have achieved strong naturalness and zero-shot voice cloning performance, but fine-grained control of expressive speech at the word or phoneme l…
Phone Segmentation and Recognition through Phonological Activation Mapping
Shikhar Bharadwaj, Kwanghee Choi, Stephen McIntosh +8
Phone segmentation and recognition are inherently related tasks, yet modern approaches typically model them separately. We argue that phonetic structure is already latent in the re…
Linear Script Representations in Speech Foundation Models Enable Zero-Shot Transliteration
Ryan Soh-Eun Shim, Kwanghee Choi, Kalvin Chang +8
Multilingual speech foundation models such as Whisper are trained on web-scale data, where data for each language consists of a myriad of regional varieties. However, different reg…