Quasi-Orthogonality and Polysemy: The Limits of Quantum Analogy in Large Language Models

Abstract

We analyze two geometric aspects of meaning-sensitive behavior in next-token models. First, high-dimensional geometry permits exponentially many nearly orthogonal feature directions at a fixed tolerance, although their residual overlaps limit simultaneous readout. Second, polysemy requires us to distinguish classical convex mixtures of sense vectors, context-dependent occurrence vectors, quantum coherence, and an optional shared-projector incidence construction over sense-labelled bases. For ordinary transformers, the quantum comparisons are structural and deliberately limited.

12 pages, 1 figure, totally rewritten, contribution to the ES Forum on "Quantum Thinking" (Frankfurt, Germany, EU, September 21-26, 2025)