Hierarchical Concept Geometry in Language Models Emerges from Word Co-occurrence

Andres Nava, Matthieu Wyart

We propose a distributional theory of how hypernymy -- the ``is-a'' relation between general and specific concepts -- is encoded geometrically in language representations. Starting…