9 citations · 17 across the 10 of their papers we have counts for
4 papers · 1 filter
Softmax Linear: Transformers may learn to classify in-context by kernel gradient descent
Sara Dragutinović, Andrew M. Saxe, Aaditya K. Singh
The remarkable ability of transformers to learn new concepts solely by reading examples within the input prompt, termed in-context learning (ICL), is a crucial aspect of intelligen…
Distinct Computations Emerge From Compositional Curricula in In-Context Learning
Jin Hwa Lee, Andrew K. Lampinen, Aaditya K. Singh +1
In-context learning (ICL) research often considers learning a function in-context through a uniform sample of input-output pairs. Here, we investigate how presenting a compositiona…
Strategy Coopetition Explains the Emergence and Transience of In-Context Learning
Aaditya K. Singh, Ted Moskovitz, Sara Dragutinovic +3
In-context learning (ICL) is a powerful ability that emerges in transformer models, enabling them to learn from context without weight updates. Recent work has established emergent…
Training Dynamics of In-Context Learning in Linear Attention
Yedi Zhang, Aaditya K. Singh, Peter E. Latham +1
While attention-based models have demonstrated the remarkable ability of in-context learning (ICL), the theoretical understanding of how these models acquired this ability through…