13 citations · 17 across the 8 of their papers we have counts for
3 papers · 1 filter
Token Erasure as a Footprint of Implicit Vocabulary Items in LLMs
Sheridan Feucht, David Atkinson, Byron Wallace +1
LLMs process text as sequences of tokens that roughly correspond to words, where less common words are represented by multiple tokens. However, individual tokens are often semantic…
Locating and Editing Factual Associations in Mamba
Arnab Sen Sharma, David Atkinson, David Bau
We investigate the mechanisms of factual recall in the Mamba state space model. Our work is inspired by previous findings in autoregressive transformer language models suggesting t…
Algorithmic progress in language models
Anson Ho, Tamay Besiroglu, Ege Erdil +6
We investigate the rate at which algorithms for pre-training language models have improved since the advent of deep learning. Using a dataset of over 200 language model evaluations…