Identifying Semantic Induction Heads to Understand In-Context Learning

Jie Ren, Qipeng Guo, Hang Yan +4

Although large language models (LLMs) have demonstrated remarkable performance, the lack of transparency in their inference logic raises concerns about their trustworthiness. To ga…