Mitigating Copy Bias in In-Context Learning through Neuron Pruning

Ameen Ali, Lior Wolf, Ivan Titov

Large language models (LLMs) have demonstrated impressive few-shot in-context learning (ICL) abilities. Still, we show that they are sometimes prone to a `copying bias', where they…