1 paper · 1 filter
Einar Urdshals, Jasmina Urdshals
We study how a one-layer attention-only transformer develops relevant structures while learning to sort lists of numbers. At the end of training, the model organizes its attention…