2 citations · 2 across the 10 of their papers we have counts for
7 papers · 1 filter
Vector Arithmetic in Concept and Token Subspaces
Sheridan Feucht, Byron Wallace, David Bau
In order to predict the next token, LLMs must represent semantic and surface-level information about the current word. Previous work identified two types of attention heads that di…
In-Context Learning Beyond Copying: A Training-Time Analysis of Abstractive ICL
Kerem Sahin, Sheridan Feucht, Adam Belfki +4
The sudden emergence of induction circuits and the ability to perform in-context copying in LLMs coincides with a sharp drop in training loss over a short span of the training proc…
The Dual-Route Model of Induction
Sheridan Feucht, Eric Todd, Byron Wallace +1
Prior work on in-context copying has shown the existence of induction heads, which attend to and promote individual tokens during copying. In this work we discover a new type of in…
Erasing Conceptual Knowledge from Language Models
Rohit Gandikota, Sheridan Feucht, Samuel Marks +1
In this work, we introduce Erasure of Language Memory (ELM), a principled approach to concept-level unlearning that operates by matching distributions defined by the model's own in…
Token Erasure as a Footprint of Implicit Vocabulary Items in LLMs
Sheridan Feucht, David Atkinson, Byron Wallace +1
LLMs process text as sequences of tokens that roughly correspond to words, where less common words are represented by multiple tokens. However, individual tokens are often semantic…
NEWTS: A Corpus for News Topic-Focused Summarization
Seyed Ali Bahrainian, Sheridan Feucht, Carsten Eickhoff
Text summarization models are approaching human levels of fidelity. Existing benchmarking corpora provide concordant pairs of full and abridged versions of Web, news or, profession…
A Novel Corpus of Discourse Structure in Humans and Computers
Babak Hemmatian, Sheridan Feucht, Rachel Avram +7
We present a novel corpus of 445 human- and computer-generated documents, comprising about 27,000 clauses, annotated for semantic clause types and coherence relations that allow fo…