1 citations · 1 across the 2 of their papers we have counts for
2 papers
cs.LG2026
Linear Predictability of Attention Heads in Large Language Models
Khalid Shaikh, Asmit Kumar Singh, Rebecca Christopher Dsouza +1
Large language model (LLM) inference is increasingly bottlenecked by the Key-Value (KV) cache, yet the fine-grained structure of attention-head activations remains poorly understoo…
cs.LG2024★ 1 cited
Unveiling and Harnessing Hidden Attention Sinks: Enhancing Large Language Models without Training through Attention Calibration
Zhongzhi Yu, Zheng Wang, Yonggan Fu +3
Attention is a fundamental component behind the remarkable achievements of large language models (LLMs). However, our current understanding of the attention mechanism, especially r…