1 citations · 1 across the 13 of their papers we have counts for
7 papers · 1 filter
Copy-Paste Language Models
Nathan Godey, Yoav Artzi
Transformers are intuitively considered to be efficient at copying from their inputs. We actually show that the ability of LLMs to copy in context can be improved through explicit…
The Surprising Effectiveness of Shared Memory in Looped Transformers
Giovanni Monea, Keshav Ramji, Yousef El-Kurdi +4
Looped Transformers apply the same layers several times per token, adding compute to improve quality without more parameters. Each recursion, however, writes its own key-value cach…
Almost Free State Prediction Separation
John Langford, Nathan Godey, Giovanni Monea +5
State--prediction separation (SPS) relieves a language model's hidden state of two competing burdens---summarizing the context and predicting the next token---by splitting the forw…
Matryoshka Language Model Suites
Nathan Godey, Yoav Artzi
Training a language model suite classically requires training each model separately and serving them independently. We improve both training and inference efficiency by stacking su…
Co-LMLM: Continuous-Query Limited Memory Language Models
Yair Feldman, Linxi Zhao, Nathan Godey +5
Externalizing knowledge in LLM pre-training is a promising avenue to achieve higher performance at smaller scales, control knowledge use, and overall increase model transparency. W…
The State-Prediction Separation Hypothesis
Giovanni Monea, Nathan Godey, Kianté Brantley +1
Transformers use the same forward computation stream to both predict the next token and store useful state for future token predictions. We formulate the \emph{state-prediction sep…
Lost in Backpropagation: The LM Head is a Gradient Bottleneck
Nathan Godey, Yoav Artzi
The last layer of neural language models (LMs) projects output features of dimension to logits in dimension , the size of the vocabulary, where usually . This misma…