Fine-Tuning Dynamics of In-Context Factual Recall in Transformers

Ruomin Huang, Eshaan Nichani, Jason D. Lee +1

In-context learning \ -- performing tasks based on examples given in the prompt \ -- is an important capability that has emerged in large language models and has received significa…