3.7k citations
- Meta (United States)US158 papers
- New York UniversityUS11 papers
- Carnegie Mellon UniversityUS10 papers
- University of California, BerkeleyUS8 papers
- Stanford UniversityUS7 papers
- University of WashingtonUS7 papers
- Cornell UniversityUS6 papers
- University of OxfordGB6 papers
- Massachusetts Institute of TechnologyUS5 papers
- Meta (Israel)IL5 papers
- ETH ZurichCH4 papers
- Google (United States)US4 papers
24 papers · 2 filters
Libri-Light: A Benchmark for ASR with Limited or No Supervision
Jacob Kahn, Morgane Rivière, Weiyi Zheng +12
We introduce a new collection of spoken English audio suitable for training speech recognition systems under limited or no supervision. It is derived from open-source audio books f…
End-to-end ASR: from Supervised to Semi-Supervised Learning with Modern Architectures
Gabriel Synnaeve, Qiantong Xu, Jacob Kahn +6
We study pseudo-labeling for the semi-supervised training of ResNet, Time-Depth Separable ConvNets, and Transformers for speech recognition, with either CTC or Seq2Seq loss functio…
Effectiveness of self-supervised pre-training for speech recognition
Alexei Baevski, Michael Auli, Abdelrahman Mohamed
We compare self-supervised representation learning algorithms which either explicitly quantize the audio data or learn representations without quantization. We find the former to b…
Blockwise Self-Attention for Long Document Understanding
Jiezhong Qiu, Hao Ma, Omer Levy +3
We present BlockBERT, a lightweight and efficient BERT model for better modeling long-distance dependencies. Our model extends BERT by introducing sparse block structures into the…
Emerging Cross-lingual Structure in Pretrained Language Models
Shijie Wu, Alexis Conneau, Haoran Li +2
We study the problem of multilingual masked language modeling, i.e. the training of a single model on concatenated text from multiple languages, and present a detailed study of sev…
RTFM: Generalising to Novel Environment Dynamics via Reading
Victor Zhong, Tim Rocktäschel, Edward Grefenstette
Obtaining policies that can generalise to new environments in reinforcement learning is challenging. In this work, we demonstrate that language understanding via a reading policy l…