318 citations · 1.3k across the 22 of their papers we have counts for
6 papers · 1 filter
Reservoir Transformers
Sheng Shen, Alexei Baevski, Ari S. Morcos +3
We demonstrate that transformers obtain impressive performance even when some of the layers are randomly initialized and never updated. Inspired by old and well-established ideas i…
The Zero Resource Speech Benchmark 2021: Metrics and baselines for unsupervised spoken language modeling
Tu Anh Nguyen, Maureen de Seyssel, Patricia Rozé +5
We introduce a new unsupervised task, spoken language modeling: the learning of linguistic representations from raw audio signals without any labels, along with the Zero Resource S…
A Comparison of Discrete Latent Variable Models for Speech Representation Learning
Henry Zhou, Alexei Baevski, Michael Auli
Neural latent variable models enable the discovery of interesting structure in speech audio data. This paper presents a comparison of two different approaches which are broadly bas…
Self-training and Pre-training are Complementary for Speech Recognition
Qiantong Xu, Alexei Baevski, Tatiana Likhomanenko +5
Self-training and unsupervised pre-training have emerged as effective approaches to improve speech recognition systems using unlabeled data. However, it is not clear whether they l…
Unsupervised Cross-lingual Representation Learning for Speech Recognition
Alexis Conneau, Alexei Baevski, Ronan Collobert +2
This paper presents XLSR which learns cross-lingual speech representations by pretraining a single model from the raw waveform of speech in multiple languages. We build on wav2vec…
wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations
Alexei Baevski, Henry Zhou, Abdelrahman Mohamed +1
We show for the first time that learning powerful representations from speech audio alone followed by fine-tuning on transcribed speech can outperform the best semi-supervised meth…