The Effect of Training Task Diversity on In-Context Learning through the Lens of Low-Dimensional Subspaces

Soo Min Kwon, Alec S. Xu, Can Yaras +3

The transformer's emergent ability to perform in-context learning (ICL) has sparked a wide range of studies designed to understand its underlying mechanisms. Existing works often s…