4k citations · 13.4k across the 76 of their papers we have counts for
5 papers · 2 filters
Libri-Light: A Benchmark for ASR with Limited or No Supervision
Jacob Kahn, Morgane Rivière, Weiyi Zheng +12
We introduce a new collection of spoken English audio suitable for training speech recognition systems under limited or no supervision. It is derived from open-source audio books f…
CCNet: Extracting High Quality Monolingual Datasets from Web Crawl Data
Guillaume Wenzek, Marie-Anne Lachaux, Alexis Conneau +4
Pre-training text representations have led to significant improvements in many areas of natural language processing. The quality of these models benefits greatly from the size of t…
CCMatrix: Mining Billions of High-Quality Parallel Sentences on the WEB
Holger Schwenk, Guillaume Wenzek, Sergey Edunov +2
We show that margin-based bitext mining in a multilingual sentence space can be applied to monolingual corpora of billions of sentences. We are using ten snapshots of a curated com…
Updating Pre-trained Word Vectors and Text Classifiers using Monolingual Alignment
Piotr Bojanowski, Onur Celebi, Tomas Mikolov +2
In this paper, we focus on the problem of adapting word vector-based models to new textual data. Given a model pre-trained on large reference data, how can we adapt it to a smaller…
Cooperative Learning of Disjoint Syntax and Semantics
Serhii Havrylov, Germán Kruszewski, Armand Joulin
There has been considerable attention devoted to models that learn to jointly infer an expression's syntactic structure and its semantics. Yet, \citet{NangiaB18} has recently shown…