4k citations · 13.4k across the 76 of their papers we have counts for
11 papers · 1 filter
Libri-Light: A Benchmark for ASR with Limited or No Supervision
Jacob Kahn, Morgane Rivière, Weiyi Zheng +12
We introduce a new collection of spoken English audio suitable for training speech recognition systems under limited or no supervision. It is derived from open-source audio books f…
CCNet: Extracting High Quality Monolingual Datasets from Web Crawl Data
Guillaume Wenzek, Marie-Anne Lachaux, Alexis Conneau +4
Pre-training text representations have led to significant improvements in many areas of natural language processing. The quality of these models benefits greatly from the size of t…
CCMatrix: Mining Billions of High-Quality Parallel Sentences on the WEB
Holger Schwenk, Guillaume Wenzek, Sergey Edunov +2
We show that margin-based bitext mining in a multilingual sentence space can be applied to monolingual corpora of billions of sentences. We are using ten snapshots of a curated com…
Updating Pre-trained Word Vectors and Text Classifiers using Monolingual Alignment
Piotr Bojanowski, Onur Celebi, Tomas Mikolov +2
In this paper, we focus on the problem of adapting word vector-based models to new textual data. Given a model pre-trained on large reference data, how can we adapt it to a smaller…
Reducing Transformer Depth on Demand with Structured Dropout
Angela Fan, Edouard Grave, Armand Joulin
Overparameterized transformer networks have obtained state of the art results in various natural language processing tasks, such as machine translation, language modeling, and ques…
Why Build an Assistant in Minecraft?
Arthur Szlam, Jonathan Gray, Kavya Srinet +11
In this document we describe a rationale for a research program aimed at building an open "assistant" in the game Minecraft, in order to make progress on the problems of natural la…