most citedSmall-scale proxies for large-scale Transformer training instabilities

4 citations · 5 across the 3 of their papers we have counts for

collaborators
Showing cs.CLShow all

2 papers · 1 filter

cs.CL2024

Training LLMs over Neurally Compressed Text

Brian Lester, Jaehoon Lee, Alex Alemi +4

In this paper, we explore the idea of training large language models (LLMs) over highly compressed text. While standard subword tokenizers compress text by a small factor, neural t…

cs.CL2023

Frontier Language Models are not Robust to Adversarial Arithmetic, or "What do I need to say so you agree 2+2=5?

C. Daniel Freeman, Laura Culp, Aaron Parisi +27

We introduce and study the problem of adversarial arithmetic, which provides a simple yet challenging testbed for language model alignment. This problem is comprised of arithmetic…