Polynomial Regression as a Task for Understanding In-context Learning Through Finetuning and Alignment

Max Wilcoxson, Morten Svendgård, Ria Doshi +3

Simple function classes have emerged as toy problems to better understand in-context-learning in transformer-based architectures used for large language models. But previously prop…