Decomposing Prediction Mechanisms for In-Context Recall

Sultan Daniels, Dylan Davis, Dhruv Gautam +3

We introduce a new family of toy problems that combine features of linear-regression-style continuous in-context learning (ICL) with discrete associative recall. We pretrain transf…