most citedIn-Context Learning of a Linear Transformer Block: Benefits of the MLP Component and One-Step GD Initialization

1 citations · 2 across the 5 of their papers we have counts for

collaborators
Showing cs.LGShow all

2 papers · 1 filter

cs.LG2025

Improved Scaling Laws in Linear Regression via Data Reuse

Licong Lin, Jingfeng Wu, Peter L. Bartlett

Neural scaling laws suggest that the test error of large language models trained online decreases polynomially as the model size and data size increase. However, such scaling can b…

cs.LG2024

Context-Scaling versus Task-Scaling in In-Context Learning

Amirhesam Abedsoltan, Adityanarayanan Radhakrishnan, Jingfeng Wu +1

Transformers exhibit In-Context Learning (ICL), where these models solve new tasks by using examples in the prompt without additional training. In our work, we identify and analyze…