1k citations · 3.2k across the 73 of their papers we have counts for
88 papers
Guess & Sketch: Language Model Guided Transpilation
Celine Lee, Abdulrahman Mahmoud, Michal Kurek +5
Maintaining legacy software requires many software and systems engineering hours. Assembly code programs, which demand low-level control over the computer machine state and have no…
OBELICS: An Open Web-Scale Filtered Dataset of Interleaved Image-Text Documents
Hugo Laurençon, Lucile Saulnier, Léo Tronchon +9
Large multimodal models trained on natural documents, which interleave images and text, outperform models trained on image-text pairs on various multimodal benchmarks. However, the…
Abductive Commonsense Reasoning Exploiting Mutually Exclusive Explanations
Wenting Zhao, Justin T. Chiu, Claire Cardie +1
Abductive reasoning aims to find plausible explanations for an event. This style of reasoning is critical for commonsense tasks where there are often multiple plausible explanation…
HOP, UNION, GENERATE: Explainable Multi-hop Reasoning without Rationale Supervision
Wenting Zhao, Justin T. Chiu, Claire Cardie +1
Explainable multi-hop question answering (QA) not only predicts answers but also identifies rationales, i. e. subsets of input sentences used to derive the answers. This problem ha…
Scaling Data-Constrained Language Models
Niklas Muennighoff, Alexander M. Rush, Boaz Barak +6
The current trend of scaling language models involves increasing both parameter count and training dataset size. Extrapolating this trend suggests that training dataset size may so…
Pretraining Without Attention
Junxiong Wang, Jing Nathan Yan, Albert Gu +1
Transformers have been essential to pretraining success in NLP. While other architectures have been used, downstream accuracy is either significantly worse, or requires attention l…