The Curved Spacetime of Transformer Architectures

Riccardo Di Sipio, Jairo Diaz-Rodriguez, Luis Serrano

We present a geometric framework for understanding Transformer-based language models, drawing an explicit analogy to General Relativity. Queries and keys induce an effective metric…