Attention Heads of Large Language Models: A Survey

Zifan Zheng, Yezhaohui Wang, Yuxin Huang +5

Since the advent of ChatGPT, Large Language Models (LLMs) have excelled in various tasks but remain as black-box systems. Understanding the reasoning bottlenecks of LLMs has become…