3.6k citations · 5.2k across the 39 of their papers we have counts for
45 papers
Compressing Streaming Neural Audio Encoders via Latent-Space Distillation
Prasanth Yadla, Mohammad Samragh, Dongseong Hwang +7
System-wide Dictation on Apple devices runs entirely on-device, and the speech it transcribes reaches the foundation model through a tokenizer: an encoder that maps short windows o…
Memory Efficient Audio Synthesis with Decoupled Temporal Depth Diffusion Transformers
Dongseong Hwang, Prasanth Yadla, Kaan Elgin +8
Siri Expressive Voices synthesize rich, configurable speech in real time and entirely on device, powered by AFM 3 Core Advanced, Apple's most powerful on-device foundation model. T…
SLM: Bridge the thin gap between speech and text foundation models
Mingqiu Wang, Wei Han, Izhak Shafran +15
We present a joint Speech and Language Model (SLM), a multitask, multilingual, and dual-modal model that takes advantage of pretrained foundational speech and language models. SLM…
Efficient Adapters for Giant Speech Models
Nanxin Chen, Izhak Shafran, Yu Zhang +4
Large pre-trained speech models are widely used as the de-facto paradigm, especially in scenarios when there is a limited amount of labeled data available. However, finetuning all…
Google USM: Scaling Automatic Speech Recognition Beyond 100 Languages
Yu Zhang, Wei Han, James Qin +24
We introduce the Universal Speech Model (USM), a single large model that performs automatic speech recognition (ASR) across 100+ languages. This is achieved by pre-training the enc…
Textless Direct Speech-to-Speech Translation with Discrete Speech Representation
Xinjian Li, Ye Jia, Chung-Cheng Chiu
Research on speech-to-speech translation (S2ST) has progressed rapidly in recent years. Many end-to-end systems have been proposed and show advantages over conventional cascade sys…
Accented Speech Recognition: Benchmarking, Pre-training, and Diverse Data
Alëna Aksënova, Zhehuai Chen, Chung-Cheng Chiu +8
Building inclusive speech recognition systems is a crucial step towards developing technologies that speakers of all language varieties can use. Therefore, ASR systems must work fo…
Self-supervised Learning with Random-projection Quantizer for Speech Recognition
Chung-Cheng Chiu, James Qin, Yu Zhang +2
We present a simple and effective self-supervised learning approach for speech recognition. The approach learns a model to predict the masked speech signals, in the form of discret…
Cross-attention conformer for context modeling in speech enhancement for ASR
Arun Narayanan, Chung-Cheng Chiu, Tom O'Malley +2
This work introduces \emph{cross-attention conformer}, an attention-based architecture for context modeling in speech enhancement. Given that the context information can often be s…
W2v-BERT: Combining Contrastive Learning and Masked Language Modeling for Self-Supervised Speech Pre-Training
Yu-An Chung, Yu Zhang, Wei Han +4
Motivated by the success of masked language modeling~(MLM) in pre-training natural language processing models, we propose w2v-BERT that explores MLM for self-supervised speech repr…
BigSSL: Exploring the Frontier of Large-Scale Semi-Supervised Learning for Automatic Speech Recognition
Yu Zhang, Daniel S. Park, Wei Han +23
We summarize the results of a host of efforts using giant automatic speech recognition (ASR) models pre-trained using large, diverse unlabeled datasets containing approximately a m…
Bridging the gap between streaming and non-streaming ASR systems bydistilling ensembles of CTC and RNN-T models
Thibault Doutre, Wei Han, Chung-Cheng Chiu +3
Streaming end-to-end automatic speech recognition (ASR) systems are widely used in everyday applications that require transcribing speech to text in real-time. Their minimal latenc…
Pushing the Limits of Non-Autoregressive Speech Recognition
Edwin G. Ng, Chung-Cheng Chiu, Yu Zhang +1
We combine recent advancements in end-to-end speech recognition to non-autoregressive automatic speech recognition. We push the limits of non-autoregressive state-of-the-art result…
Efficient Knowledge Distillation for RNN-Transducer Models
Sankaran Panchapagesan, Daniel S. Park, Chung-Cheng Chiu +3
Knowledge Distillation is an effective method of transferring knowledge from a large model to a smaller model. Distillation can be viewed as a type of model compression, and has pl…
A Better and Faster End-to-End Model for Streaming ASR
Bo Li, Anmol Gulati, Jiahui Yu +12
End-to-end (E2E) models have shown to outperform state-of-the-art conventional models for streaming speech recognition [1] across many dimensions, including quality (as measured by…
Cascaded encoders for unifying streaming and non-streaming ASR
Arun Narayanan, Tara N. Sainath, Ruoming Pang +5
End-to-end (E2E) automatic speech recognition (ASR) models, by now, have shown competitive performance on several benchmarks. These models are structured to either operate in strea…
Improving Streaming Automatic Speech Recognition With Non-Streaming Model Distillation On Unsupervised Data
Thibault Doutre, Wei Han, Min Ma +7
Streaming end-to-end automatic speech recognition (ASR) models are widely used on smart speakers and on-device applications. Since these models are expected to transcribe speech wi…
FastEmit: Low-latency Streaming ASR with Sequence-level Emission Regularization
Jiahui Yu, Chung-Cheng Chiu, Bo Li +8
Streaming automatic speech recognition (ASR) aims to emit each hypothesized word as quickly and accurately as possible. However, emitting fast without degrading quality, as measure…
Pushing the Limits of Semi-Supervised Learning for Automatic Speech Recognition
Yu Zhang, James Qin, Daniel S. Park +5
We employ a combination of recent developments in semi-supervised learning for automatic speech recognition to obtain state-of-the-art results on LibriSpeech utilizing the unlabele…
Dual-mode ASR: Unify and Improve Streaming ASR with Full-context Modeling
Jiahui Yu, Wei Han, Anmol Gulati +5
Streaming automatic speech recognition (ASR) aims to emit each hypothesized word as quickly and accurately as possible, while full-context ASR waits for the completion of a full sp…
Parallel Rescoring with Transformer for Streaming On-Device Speech Recognition
Wei Li, James Qin, Chung-Cheng Chiu +2
Recent advances of end-to-end models have outperformed conventional models through employing a two-pass model. The two-pass model provides better speed-quality trade-offs for on-de…
Conformer: Convolution-augmented Transformer for Speech Recognition
Anmol Gulati, James Qin, Chung-Cheng Chiu +8
Recently Transformer and Convolution neural network (CNN) based models have shown promising results in Automatic Speech Recognition (ASR), outperforming Recurrent neural networks (…
ContextNet: Improving Convolutional Neural Networks for Automatic Speech Recognition with Global Context
Wei Han, Zhengdong Zhang, Yu Zhang +6
Convolutional neural networks (CNN) have shown promising results for end-to-end speech recognition, albeit still behind other state-of-the-art methods in performance. In this paper…
Improved Noisy Student Training for Automatic Speech Recognition
Daniel S. Park, Yu Zhang, Ye Jia +5
Recently, a semi-supervised learning method known as "noisy student training" has been shown to improve image classification performance of deep networks significantly. Noisy stude…