Retrieval Heads are Dynamic

Yuping Lin, Zitao Li, Yue Xing +6

Recent studies have identified "retrieval heads" in Large Language Models (LLMs) responsible for extracting information from input contexts. However, prior works largely rely on st…