246 citations · 333 across the 3 of their papers we have counts for
6 papers
HyperTree Proof Search for Neural Theorem Proving
Guillaume Lample, Marie-Anne Lachaux, Thibaut Lavril +5
We propose an online training procedure for a transformer-based automated theorem prover. Our approach leverages a new search algorithm, HyperTree Proof Search (HTPS), inspired by…
DOBF: A Deobfuscation Pre-Training Objective for Programming Languages
Baptiste Roziere, Marie-Anne Lachaux, Marc Szafraniec +1
Recent advances in self-supervised learning have dramatically improved the state of the art on a wide variety of tasks. However, research in language model pre-training has mostly…
Target Conditioning for One-to-Many Generation
Marie-Anne Lachaux, Armand Joulin, Guillaume Lample
Neural Machine Translation (NMT) models often lack diversity in their generated translations, even when paired with search algorithm, like beam search. A challenge is that the dive…
Unsupervised Translation of Programming Languages
Marie-Anne Lachaux, Baptiste Roziere, Lowik Chanussot +1
A transcompiler, also known as source-to-source translator, is a system that converts source code from a high-level programming language (such as C++ or Python) to another. Transco…
CCNet: Extracting High Quality Monolingual Datasets from Web Crawl Data
Guillaume Wenzek, Marie-Anne Lachaux, Alexis Conneau +4
Pre-training text representations have led to significant improvements in many areas of natural language processing. The quality of these models benefits greatly from the size of t…
Poly-encoders: Transformer Architectures and Pre-training Strategies for Fast and Accurate Multi-sentence Scoring
Samuel Humeau, Kurt Shuster, Marie-Anne Lachaux +1
The use of deep pre-trained bidirectional transformers has led to remarkable progress in a number of applications (Devlin et al., 2018). For tasks that make pairwise comparisons be…