MateICL: Mitigating Attention Dispersion in Large-Scale In-Context Learning

Murtadha Ahmed, Wenbo, Liu yunfeng

Large Language Models (LLMs) have demonstrated remarkable capabilities in In-Context Learning (ICL). However, the fixed position length constraints in pre-trained models limit the…