Attention as Conditioning: What Classical Learning Theory Predicts About Linear Transformers

Mu Qiao

Attention is widely understood as an associative memory, but that description alone does not predict how the memory will behave. Predictive theories do exist, but in the literature…