9 citations · 13 across the 11 of their papers we have counts for
4 papers · 1 filter
In-context classification: Calibration and noise in linear attention with softmax readout
Arman Rysmakhanov, Mary Letey, Yue M. Lu +2
We study in-context binary classification in a simple attention model trained to minimise a logistic loss. Each prompt has a latent logistic teacher, and the learner must infer its…
Sequential Correlations Change In-Context Learning: Effective Context Length and Architectural Mismatch
Mary Letey, Yue M. Lu, Cengiz Pehlevan +1
Modern sequence models have a striking capacity for in-context learning (ICL); they can perform new tasks based only on examples given in the prompt. Understanding how this ability…
Sharp Capacity Thresholds in Linear Associative Memory: From Top-1 Retrieval to Tail-Average Learning
Nicholas Barnfield, Juno Kim, Eshaan Nichani +2
How many key-value associations can a linear memory store? The answer depends not only on the degrees of freedom in the memory matrix, but also on the retrieval c…
Asymptotic Theory of Iterated Empirical Risk Minimization, with Applications to Active Learning
Hugo Cui, Yue M. Lu
We study a class of iterated empirical risk minimization (ERM) procedures in which two successive ERMs are performed on the same dataset, and the predictions of the first estimator…