Showing cs.CLShow all
3 papers · 1 filter
cs.CL2025
Idiosyncrasies in Large Language Models
Mingjie Sun, Yida Yin, Zhiqiu Xu +2
In this work, we unveil and study idiosyncrasies in Large Language Models (LLMs) -- unique patterns in their outputs that can be used to distinguish the models. To do so, we consid…
cs.CL2025
Massive Values in Self-Attention Modules are the Key to Contextual Knowledge Understanding
Mingyu Jin, Kai Mei, Wujiang Xu +5
Large language models (LLMs) have achieved remarkable success in contextual knowledge understanding. In this paper, we show that these concentrated massive values consistently emer…
cs.CL2024
Massive Activations in Large Language Models
Mingjie Sun, Xinlei Chen, J. Zico Kolter +1
We observe an empirical phenomenon in Large Language Models (LLMs) -- very few activations exhibit significantly larger values than others (e.g., 100,000 times larger). We call the…