The Geometry of Thought: Disclosing the Transformer as a Tropical Polynomial Circuit

Faruk Alpay, Bilge Senturk

We prove that the Transformer self-attention mechanism in the high-confidence regime ($\beta \to \infty$, where $\beta$ is an inverse temperature) operates in the tropical semiring (max-plu…