2 citations · 2 across the 10 of their papers we have counts for
3 papers · 1 filter
Vector Arithmetic in Concept and Token Subspaces
Sheridan Feucht, Byron Wallace, David Bau
In order to predict the next token, LLMs must represent semantic and surface-level information about the current word. Previous work identified two types of attention heads that di…
In-Context Learning Beyond Copying: A Training-Time Analysis of Abstractive ICL
Kerem Sahin, Sheridan Feucht, Adam Belfki +4
The sudden emergence of induction circuits and the ability to perform in-context copying in LLMs coincides with a sharp drop in training loss over a short span of the training proc…
The Dual-Route Model of Induction
Sheridan Feucht, Eric Todd, Byron Wallace +1
Prior work on in-context copying has shown the existence of induction heads, which attend to and promote individual tokens during copying. In this work we discover a new type of in…