6.9k citations
- Meta (United Kingdom)GB158 papers
- Meta (Israel)IL62 papers
- Menlo SchoolUS51 papers
- New York UniversityUS49 papers
- Carnegie Mellon UniversityUS33 papers
- University of California, BerkeleyUS31 papers
- The University of Texas at AustinUS28 papers
- Google (United States)US27 papers
- Georgia Institute of TechnologyUS26 papers
- Stanford UniversityUS25 papers
- Cornell UniversityUS24 papers
- University of WashingtonUS20 papers
38 papers · 2 filters
Libri-Light: A Benchmark for ASR with Limited or No Supervision
Jacob Kahn, Morgane Rivière, Weiyi Zheng +12
We introduce a new collection of spoken English audio suitable for training speech recognition systems under limited or no supervision. It is derived from open-source audio books f…
AIPNet: Generative Adversarial Pre-training of Accent-invariant Networks for End-to-end Speech Recognition
Yi-Chen Chen, Zhaojun Yang, Ching-Feng Yeh +2
As one of the major sources in speech variability, accents have posed a grand challenge to the robustness of speech recognition systems. In this paper, our goal is to build a unifi…
End-to-end ASR: from Supervised to Semi-Supervised Learning with Modern Architectures
Gabriel Synnaeve, Qiantong Xu, Jacob Kahn +6
We study pseudo-labeling for the semi-supervised training of ResNet, Time-Depth Separable ConvNets, and Transformers for speech recognition, with either CTC or Seq2Seq loss functio…
Effectiveness of self-supervised pre-training for speech recognition
Alexei Baevski, Michael Auli, Abdelrahman Mohamed
We compare self-supervised representation learning algorithms which either explicitly quantize the audio data or learn representations without quantization. We find the former to b…
CamemBERT: a Tasty French Language Model
Louis Martin, Benjamin Muller, Pedro Javier Ortiz Suárez +5
Pretrained language models are now ubiquitous in Natural Language Processing. Despite their success, most available models have either been trained on English data or on the concat…
Blockwise Self-Attention for Long Document Understanding
Jiezhong Qiu, Hao Ma, Omer Levy +3
We present BlockBERT, a lightweight and efficient BERT model for better modeling long-distance dependencies. Our model extends BERT by introducing sparse block structures into the…