2 papers
cs.LG2025
Decomposing Prediction Mechanisms for In-Context Recall
Sultan Daniels, Dylan Davis, Dhruv Gautam +3
We introduce a new family of toy problems that combine features of linear-regression-style continuous in-context learning (ICL) with discrete associative recall. We pretrain transf…
cs.LG2024
Polynomial Regression as a Task for Understanding In-context Learning Through Finetuning and Alignment
Max Wilcoxson, Morten Svendgård, Ria Doshi +3
Simple function classes have emerged as toy problems to better understand in-context-learning in transformer-based architectures used for large language models. But previously prop…