21 papers
Density-Robust Spherical Coordinates from Persistent Cohomology
Nick Nordwald, Inés GarcÃa-Redondo, Anthea Monod
Persistent cohomology provides a principled framework for constructing nonlinear coordinates that reflect the topology of data. However, these topological coordinates can be severe…
Tracking Representation Dynamics in Large Language Models with Persistent Homology
Naman Malhotra, Jay Ambadkar, Abhinav Gupta +4
Large language models are commonly aligned through supervised fine-tuning, yet little is known about how their internal representations evolve during this process. We study alignme…
Generative Modeling on Metric Graphs via Neural Optimal Transport
Alessandro Micheli, Yueqi Cao, Anthea Monod +1
We introduce, to our knowledge, the first deep generative modeling framework for probability distributions continuously supported on compact metric graphs. Given source and target…
Non-Archimedean Polydisc Spaces and Applications to Optimisation
Paul Lezeau, Yiannis Fam, Anthea Monod +1
We propose a new framework for optimisation over non-Archimedean spaces inspired by Berkovich geometry. Specifically, we introduce polydisc spaces, which consists of products of cl…
Topological Signatures of Grokking
Yifan Tang, Qiquan Wang, Inés GarcÃa-Redondo +1
We study the grokking phenomenon through the lens of topology. Using persistent homology on point clouds derived from the embedding matrices of a range of models trained on modular…
Feature Starvation as Geometric Instability in Sparse Autoencoders
Faris Chaudhry, Keisuke Yano, Anthea Monod
Sparse autoencoders (SAEs) are used to disentangle the dense, polysemantic internal representations of large language models (LLMs) into interpretable, monosemantic concepts. Howev…