Investigation into In-Context Learning Capabilities of Transformers

Rushil Chandrupatla, Leo Bangayan, Sebastian Leng

Transformers have demonstrated a strong ability for in-context learning (ICL), enabling models to solve previously unseen tasks using only example input output pairs provided at in…