Momentum Point-Perplexity Mechanics in Large Language Models

Lorenzo Tomaz, Judd Rosenblatt, Thomas Berry Jones +1

We take a physics-based approach to studying how the internal hidden states of large language models change from token to token during inference. Across 20 open-source transformer…