Scale Determines Whether Language Models Organize Representation Geometry for Prediction

Weilun Xu

In language models, what a representation encodes is determined by the geometry of its representation space: distances, not activations, carry meaning. Existing tools characterize…