Transformer learns the cross-task prior and regularization for in-context learning

Fei Lu, Yue Yu

Transformers have shown a remarkable ability for in-context learning (ICL), making predictions based on contextual examples. However, while theoretical analyses have explored this…