How does Multi-Task Training Affect Transformer In-Context Capabilities? Investigations with Function Classes

Harmon Bhasin, Timothy Ossowski, Yiqiao Zhong +1

Large language models (LLM) have recently shown the extraordinary ability to perform unseen tasks based on few-shot examples provided as text, also known as in-context learning (IC…