One-Layer Transformer Provably Learns Multiclass One-Nearest Neighbor in Context

Skanda Athreya, Yutong Wang

We extend recent work establishing an equivalence between one-layer transformers and nearest-neighbor classifiers in the binary setting to the multiclass case. By leveraging the si…