461 citations · 728 across the 27 of their papers we have counts for
31 papers
A Practical Recipe for Semi-Supervised Federated ASR: Online Pseudo-Labels with Server Update Stabilization
Wonho Bae, Zakaria Aldeneh, Martin Pelikan +3
Semi-supervised federated learning (SSFL) trains models on clients' unlabeled data using a teacher to generate pseudo-labels, with a small labeled seed dataset on the server. Autom…
Path-Constrained Mixture-of-Experts
Zijin Gu, Tatiana Likhomanenko, Vimal Thilak +2
Sparse Mixture-of-Experts (MoE) architectures route each token through a subset of experts at each layer independently. We propose viewing MoE computation through the lens of \emph…
Which Data Matter? Embedding-Based Data Selection for Speech Recognition
Zakaria Aldeneh, Skyler Seto, Maureen de Seyssel +8
Modern ASR systems are typically trained on large-scale pseudo-labeled, in-the-wild data spanning multiple domains. While such heterogeneous data benefit generalist models designed…
Closing the Gap Between Text and Speech Understanding in LLMs
Santiago Cuervo, Skyler Seto, Maureen de Seyssel +5
Large Language Models (LLMs) can be adapted to extend their text capabilities to speech inputs. However, these speech-adapted LLMs consistently underperform their text-based counte…
ChipChat: Low-Latency Cascaded Conversational Agent in MLX
Tatiana Likhomanenko, Richard He Bai, Zijin Gu +8
The emergence of large language models (LLMs) has transformed spoken dialog systems, yet the optimal architecture for real-time on-device voice agents remains an open question. Whi…
Omni-Router: Sharing Routing Decisions in Sparse Mixture-of-Experts for Speech Recognition
Zijin Gu, Tatiana Likhomanenko, Navdeep Jaitly
Mixture-of-experts (MoE) architectures have expanded from language modeling to automatic speech recognition (ASR). Traditional MoE methods, such as the Switch Transformer, route ex…
SpeakStream: Streaming Text-to-Speech with Interleaved Data
Richard He Bai, Zijin Gu, Tatiana Likhomanenko +1
The latency bottleneck of traditional text-to-speech (TTS) systems fundamentally hinders the potential of streaming large language models (LLMs) in conversational AI. These TTS sys…
Mechanisms of Multimodal Synchronization: Insights from Decoder-Based Video-Text-to-Speech Synthesis
Akshita Gupta, Tatiana Likhomanenko, Karren Dai Yang +3
Unified decoder-only transformers have shown promise for multimodal generation, yet how sequence design affects synchronization across modalities with heterogeneous sampling rates…
dMel: Speech Tokenization made Simple
Richard He Bai, Tatiana Likhomanenko, Ruixiang Zhang +3
Large language models have revolutionized natural language processing by leveraging self-supervised pretraining on vast textual data. Inspired by this success, researchers have inv…
Revisiting ASR Error Correction with Specialized Models
Zijin Gu, Tatiana Likhomanenko, He Bai +3
Language models play a central role in automatic speech recognition (ASR), yet most methods rely on text-only models unaware of ASR error patterns. Recently, large language models…
AV-CPL: Continuous Pseudo-Labeling for Audio-Visual Speech Recognition
Andrew Rouditchenko, Ronan Collobert, Tatiana Likhomanenko
Audio-visual speech contains synchronized audio and visual information that provides cross-modal supervision to learn representations for both automatic speech recognition (ASR) an…
Unsupervised ASR via Cross-Lingual Pseudo-Labeling
Tatiana Likhomanenko, Loren Lugosch, Ronan Collobert
Recent work has shown that it is possible to train an automatic speech recognition (ASR) system using only unpaired audio and text. Existing unsupervised AS…
Joint Speech Transcription and Translation: Pseudo-Labeling with Out-of-Distribution Data
Mozhdeh Gheini, Tatiana Likhomanenko, Matthias Sperber +1
Self-training has been shown to be helpful in addressing data scarcity for many domains, including vision, speech, and language. Specifically, self-training, or pseudo-labeling, la…
Continuous Soft Pseudo-Labeling in ASR
Tatiana Likhomanenko, Ronan Collobert, Navdeep Jaitly +1
Continuous pseudo-labeling (PL) algorithms such as slimIPL have recently emerged as a powerful strategy for semi-supervised learning in speech recognition. In contrast with earlier…
More Speaking or More Speakers?
Dan Berrebbi, Ronan Collobert, Navdeep Jaitly +1
Self-training (ST) and self-supervised learning (SSL) methods have demonstrated strong improvements in automatic speech recognition (ASR). In spite of these advances, to the best o…
Continuous Pseudo-Labeling from the Start
Dan Berrebbi, Ronan Collobert, Samy Bengio +2
Self-training (ST), or pseudo-labeling has sparked significant interest in the automatic speech recognition (ASR) community recently because of its success in harnessing unlabeled…
Flashlight: Enabling Innovation in Tools for Machine Learning
Jacob Kahn, Vineel Pratap, Tatiana Likhomanenko +11
As the computational requirements for machine learning systems and the size and complexity of machine learning frameworks increases, essential framework innovation has become chall…
Pseudo-Labeling for Massively Multilingual Speech Recognition
Loren Lugosch, Tatiana Likhomanenko, Gabriel Synnaeve +1
Semi-supervised learning through pseudo-labeling has become a staple of state-of-the-art monolingual speech recognition systems. In this work, we extend pseudo-labeling to massivel…
Word Order Does Not Matter For Speech Recognition
Vineel Pratap, Qiantong Xu, Tatiana Likhomanenko +2
In this paper, we study training of automatic speech recognition system in a weakly supervised setting where the order of words in transcript labels of the audio training data is n…
Kaizen: Continuously improving teacher using Exponential Moving Average for semi-supervised speech recognition
Vimal Manohar, Tatiana Likhomanenko, Qiantong Xu +5
In this paper, we introduce the Kaizen framework that uses a continuously improving teacher to generate pseudo-labels for semi-supervised speech recognition (ASR). The proposed app…
CAPE: Encoding Relative Positions with Continuous Augmented Positional Embeddings
Tatiana Likhomanenko, Qiantong Xu, Gabriel Synnaeve +2
Without positional information, attention-based Transformer neural networks are permutation-invariant. Absolute or relative positional embeddings are the most popular ways to feed…
Robust wav2vec 2.0: Analyzing Domain Shift in Self-Supervised Pre-Training
Wei-Ning Hsu, Anuroop Sriram, Alexei Baevski +8
Self-supervised learning of speech representations has been a very active research area but most work is focused on a single domain such as read audio books for which there exist l…
Joint Masked CPC and CTC Training for ASR
Chaitanya Talnikar, Tatiana Likhomanenko, Ronan Collobert +1
Self-supervised learning (SSL) has shown promise in learning representations of audio that are useful for automatic speech recognition (ASR). But, training SSL models like wav2vec~…
Self-training and Pre-training are Complementary for Speech Recognition
Qiantong Xu, Alexei Baevski, Tatiana Likhomanenko +5
Self-training and unsupervised pre-training have emerged as effective approaches to improve speech recognition systems using unlabeled data. However, it is not clear whether they l…