4 papers
TopoFisher: Learning Topological Summary Statistics by Maximizing Fisher Information
Matteo Biagetti, Mathieu Carrière, Francesco Conti +3
Persistence diagrams provide stable, interpretable summaries of geometric and topological structure and are useful for simulation-based inference when low-order statistics miss key…
Probing Geometry of Next Token Prediction Using Cumulant Expansion of the Softmax Entropy
Karthik Viswanathan, Sang Eon Park
We introduce a cumulant-expansion framework for quantifying how large language models (LLMs) internalize higher-order statistical structure during next-token prediction. By treatin…
Persistent Topological Features in Large Language Models
Yuri Gardinazzi, Karthik Viswanathan, Giada Panerai +3
Understanding the decision-making processes of large language models is critical given their widespread applications. To achieve this, we aim to connect a formal mathematical frame…
The Intrinsic Dimension of Prompts in Internal Representations of Large Language Models
Karthik Viswanathan, Yuri Gardinazzi, Giada Panerai +2
We study the geometry of token representations at the prompt level in large language models through the lens of intrinsic dimension. Viewing transformers as mean-field particle sys…