5 papers
Which Data Matter? Embedding-Based Data Selection for Speech Recognition
Zakaria Aldeneh, Skyler Seto, Maureen de Seyssel +8
Modern ASR systems are typically trained on large-scale pseudo-labeled, in-the-wild data spanning multiple domains. While such heterogeneous data benefit generalist models designed…
Closing the Gap Between Text and Speech Understanding in LLMs
Santiago Cuervo, Skyler Seto, Maureen de Seyssel +5
Large Language Models (LLMs) can be adapted to extend their text capabilities to speech inputs. However, these speech-adapted LLMs consistently underperform their text-based counte…
Speech Foundation Models Generalize to Time Series Tasks from Wearable Sensor Data
Jaya Narain, Zakaria Aldeneh, Shirley Ren
Both speech and sensor time series data encode information in both the time- and frequency- domains, like spectral powers and waveform shapelets. We show that speech foundation mod…
Leveraging Audio-Visual Data to Reduce the Multilingual Gap in Self-Supervised Speech Models
MarÃa Andrea Cruz Blandón, Zakaria Aldeneh, Jie Chi +1
Self-supervised learning (SSL) has made significant advances in speech representation learning. Models like wav2vec 2.0 and HuBERT have achieved state-of-the-art results in tasks s…
ChipChat: Low-Latency Cascaded Conversational Agent in MLX
Tatiana Likhomanenko, Luke Carlson, Richard He Bai +8
The emergence of large language models (LLMs) has transformed spoken dialog systems, yet the optimal architecture for real-time on-device voice agents remains an open question. Whi…