Showing cs.CLShow all
3 papers · 1 filter
cs.CL2025
Probing Geometry of Next Token Prediction Using Cumulant Expansion of the Softmax Entropy
Karthik Viswanathan, Sang Eon Park
We introduce a cumulant-expansion framework for quantifying how large language models (LLMs) internalize higher-order statistical structure during next-token prediction. By treatin…
cs.CL2025
Persistent Topological Features in Large Language Models
Yuri Gardinazzi, Karthik Viswanathan, Giada Panerai +3
Understanding the decision-making processes of large language models is critical given their widespread applications. To achieve this, we aim to connect a formal mathematical frame…
cs.CL2025
The Intrinsic Dimension of Prompts in Internal Representations of Large Language Models
Karthik Viswanathan, Yuri Gardinazzi, Giada Panerai +2
We study the geometry of token representations at the prompt level in large language models through the lens of intrinsic dimension. Viewing transformers as mean-field particle sys…