4 citations · 7 across the 7 of their papers we have counts for
4 papers · 2 filters
Softmax Linear: Transformers may learn to classify in-context by kernel gradient descent
Sara Dragutinović, Andrew M. Saxe, Aaditya K. Singh
The remarkable ability of transformers to learn new concepts solely by reading examples within the input prompt, termed in-context learning (ICL), is a crucial aspect of intelligen…
Distinct Computations Emerge From Compositional Curricula in In-Context Learning
Jin Hwa Lee, Andrew K. Lampinen, Aaditya K. Singh +1
In-context learning (ICL) research often considers learning a function in-context through a uniform sample of input-output pairs. Here, we investigate how presenting a compositiona…
Strategy Coopetition Explains the Emergence and Transience of In-Context Learning
Aaditya K. Singh, Ted Moskovitz, Sara Dragutinovic +3
In-context learning (ICL) is a powerful ability that emerges in transformer models, enabling them to learn from context without weight updates. Recent work has established emergent…
Nonlinear dynamics of localization in neural receptive fields
Leon Lufkin, Andrew M. Saxe, Erin Grant
Localized receptive fields -- neurons that are selective for certain contiguous spatiotemporal features of their input -- populate early sensory regions of the mammalian brain. Uns…