4 citations · 7 across the 16 of their papers we have counts for
1 paper · 2 filters
Xiangming Gu, Tianyu Pang, Chao Du +5
Language Models (LMs) assign significant attention to the first token, even if it is not semantically important, which is known as attention sink. This phenomenon has been widely a…