Transformers through the lens of support-preserving maps between measures

Takashi Furuya, Maarten V. de Hoop, Matti Lassas

Transformers are deep architectures that define ``in-context maps'' which enable predicting new tokens based on a given set of tokens (such as a prompt in NLP applications or a set…