On Understanding Attention-Based In-Context Learning for Categorical Data

Aaron T. Wang, William Convertino, Xiang Cheng +2

In-context learning based on attention models is examined for data with categorical outcomes, with inference in such models viewed from the perspective of functional gradient desce…