The Stepwise Informativeness Assumption: Why are Entropy Dynamics and Reasoning Correlated in LLMs?

Mar Gonzàlez I Català, Haitz Sáez de Ocáriz Borde, George D. Montañez +1

Recent work uses entropy-based signals at multiple representation levels to study reasoning in large language models, but the field remains largely empirical. A central unresolved…