3 citations · 3 across the 8 of their papers we have counts for
Showing 2024Show all
3 papers · 1 filter
cs.LG2024
How Data Inter-connectivity Shapes LLMs Unlearning: A Structural Unlearning Perspective
Xinchi Qiu, William F. Shen, Yihong Chen +4
While unlearning knowledge from large language models (LLMs) is receiving increasing attention, one important aspect remains unexplored. Existing approaches and benchmarks assume d…
cs.CL2024★ 3 cited
Know When To Stop: A Study of Semantic Drift in Text Generation
Ava Spataru, Eric Hambro, Elena Voita +1
In this work, we explicitly show that modern LLMs tend to generate correct facts first, then "drift away" and generate incorrect facts later: this was occasionally observed but nev…
cs.AI2024
Spectral Filters, Dark Signals, and Attention Sinks
Nicola Cancedda
Projecting intermediate representations onto the vocabulary is an increasingly popular interpretation tool for transformer-based LLMs, also known as the logit lens. We propose a qu…