1.1k citations · 1.2k across the 13 of their papers we have counts for
1 paper · 1 filter
Baptiste Roziere, Marie-Anne Lachaux, Marc Szafraniec +1
Recent advances in self-supervised learning have dramatically improved the state of the art on a wide variety of tasks. However, research in language model pre-training has mostly…