261 citations · 588 across the 13 of their papers we have counts for
14 papers
Efficient Speech Representation Learning with Low-Bit Quantization
Ching-Feng Yeh, Wei-Ning Hsu, Paden Tomasello +1
With the development of hardware for machine learning, newer models often come at the cost of both increased sizes and computational complexity. In effort to improve the efficiency…
Continual Learning for On-Device Speech Recognition using Disentangled Conformers
Anuj Diwan, Ching-Feng Yeh, Wei-Ning Hsu +4
Automatic speech recognition research focuses on training and evaluating on static datasets. Yet, as speech models are increasingly deployed on personal devices, such models encoun…
Massively Multilingual ASR on 70 Languages: Tokenization, Architecture, and Generalization Capabilities
Andros Tjandra, Nayan Singhal, David Zhang +4
End-to-end multilingual ASR has become more appealing because of several reasons such as simplifying the training and deployment process and positive performance transfer from high…
Biased Self-supervised learning for ASR
Florian L. Kreyssig, Yangyang Shi, Jinxi Guo +3
Self-supervised learning via masked prediction pre-training (MPPT) has shown impressive performance on a range of speech-processing tasks. This paper proposes a method to bias self…
Evidence of Vocal Tract Articulation in Self-Supervised Learning of Speech
Cheol Jun Cho, Peter Wu, Abdelrahman Mohamed +1
Recent self-supervised learning (SSL) models have proven to learn rich representations of speech, which can readily be utilized by diverse downstream tasks. To understand such util…
Learning Lip-Based Audio-Visual Speaker Embeddings with AV-HuBERT
Bowen Shi, Abdelrahman Mohamed, Wei-Ning Hsu
This paper investigates self-supervised pre-training for audio-visual speaker representation learning where a visual stream showing the speaker's mouth area is used alongside speec…
Unified Speech-Text Pre-training for Speech Translation and Recognition
Yun Tang, Hongyu Gong, Ning Dong +8
We describe a method to jointly pre-train speech and text in an encoder-decoder modeling framework for speech translation and recognition. The proposed method incorporates four sel…
Federated Learning with Partial Model Personalization
Krishna Pillutla, Kshitiz Malik, Abdelrahman Mohamed +3
We consider two federated learning algorithms for training partially personalized models, where the shared and personal parameters are updated either simultaneously or alternately…
Generative Spoken Dialogue Language Modeling
Tu Anh Nguyen, Eugene Kharitonov, Jade Copet +8
We introduce dGSLM, the first "textless" model able to generate audio samples of naturalistic spoken dialogues. It uses recent work on unsupervised spoken unit discovery coupled wi…
HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units
Wei-Ning Hsu, Benjamin Bolte, Yao-Hung Hubert Tsai +3
Self-supervised approaches for speech representation learning are challenged by three unique problems: (1) there are multiple sound units in each input utterance, (2) there is no l…
Kaizen: Continuously improving teacher using Exponential Moving Average for semi-supervised speech recognition
Vimal Manohar, Tatiana Likhomanenko, Qiantong Xu +5
In this paper, we introduce the Kaizen framework that uses a continuously improving teacher to generate pseudo-labels for semi-supervised speech recognition (ASR). The proposed app…
BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension
Mike Lewis, Yinhan Liu, Naman Goyal +5
We present BART, a denoising autoencoder for pretraining sequence-to-sequence models. BART is trained by (1) corrupting text with an arbitrary noising function, and (2) learning a…
Training ASR models by Generation of Contextual Information
Kritika Singh, Dmytro Okhonko, Jun Liu +8
Supervised ASR models have reached unprecedented levels of accuracy, thanks in part to ever-increasing amounts of labelled training data. However, in many applications and locales,…
Transformer-based Acoustic Modeling for Hybrid Speech Recognition
Yongqiang Wang, Abdelrahman Mohamed, Duc Le +10
We propose and evaluate transformer-based acoustic models (AMs) for hybrid speech recognition. Several modeling choices are discussed in this work, including various positional emb…