A Geometric Perspective on Next-Token Prediction in Large Language Models: Three Emerging Phases

Gianfranco Lombardo, Giuseppe Trimigno, Stefano Cagnoni

We investigate the geometry of predictive information across the layers of large language models (LLMs). We repurpose representation lenses-learned affine maps trained to predict t…