Quasi-Orthogonality and Polysemy: The Limits of Quantum Analogy in Large Language Models

Karl Svozil

We analyze two geometric aspects of meaning-sensitive behavior in next-token models. First, high-dimensional geometry permits exponentially many nearly orthogonal feature direction…