62 citations · 64 across the 2 of their papers we have counts for
10 papers
Better & Faster Large Language Models via Multi-token Prediction
Fabian Gloeckle, Badr Youbi Idrissi, Baptiste Rozière +2
Large language models such as GPT and Llama are trained with a next-token prediction loss. In this work, we suggest that training language models to predict multiple future tokens…
DOBF: A Deobfuscation Pre-Training Objective for Programming Languages
Baptiste Roziere, Marie-Anne Lachaux, Marc Szafraniec +1
Recent advances in self-supervised learning have dramatically improved the state of the art on a wide variety of tasks. However, research in language model pre-training has mostly…
Black-Box Optimization Revisited: Improving Algorithm Selection Wizards through Massive Benchmarking
Laurent Meunier, Herilalaina Rakotoarison, Pak Kan Wong +5
Existing studies in black-box optimization for machine learning suffer from low generalizability, caused by a typically selective choice of problem instances used for training and…
EvolGAN: Evolutionary Generative Adversarial Networks
Baptiste Roziere, Fabien Teytaud, Vlad Hosu +4
We propose to use a quality estimator and evolutionary methods to search the latent space of generative adversarial networks trained on small, difficult datasets, or both. The new…
Tarsier: Evolving Noise Injection in Super-Resolution GANs
Baptiste Roziere, Nathanal Carraz Rakotonirina, Vlad Hosu +4
Super-resolution aims at increasing the resolution and level of detail within an image. The current state of the art in general single-image super-resolution is held by NESRGAN+, w…
Unsupervised Translation of Programming Languages
Marie-Anne Lachaux, Baptiste Roziere, Lowik Chanussot +1
A transcompiler, also known as source-to-source translator, is a system that converts source code from a high-level programming language (such as C++ or Python) to another. Transco…