99 citations · 157 across the 39 of their papers we have counts for
10 papers · 1 filter
Understanding Clustering in Slot Attention via Particle Dynamics
Vasudev Joy, Rajat Rasal, Avinash Kori +2
Studying attention through the lens of interacting particle dynamics has shown how token clustering can emerge from the underlying dynamics. We extend this perspective to slot atte…
Tracking Representation Dynamics in Large Language Models with Persistent Homology
Naman Malhotra, Jay Ambadkar, Abhinav Gupta +4
Large language models are commonly aligned through supervised fine-tuning, yet little is known about how their internal representations evolve during this process. We study alignme…
Topological Signatures of Grokking
Yifan Tang, Qiquan Wang, Inés García-Redondo +1
We study the grokking phenomenon through the lens of topology. Using persistent homology on point clouds derived from the embedding matrices of a range of models trained on modular…
Feature Starvation as Geometric Instability in Sparse Autoencoders
Faris Chaudhry, Keisuke Yano, Anthea Monod
Sparse autoencoders (SAEs) are used to disentangle the dense, polysemantic internal representations of large language models (LLMs) into interpretable, monosemantic concepts. Howev…
Breaking Symmetry Bottlenecks in GNN Readouts
Mouad Talhi, Arne Wolf, Anthea Monod
Graph neural networks (GNNs) are widely used for learning on structured data, yet their ability to distinguish non-isomorphic graphs is fundamentally limited. These limitations are…
Riemannian Neural Optimal Transport
Alessandro Micheli, Yueqi Cao, Anthea Monod +1
Computational optimal transport (OT) offers a principled framework for generative modeling. Neural OT methods, which use neural networks to learn an OT map (or potential) from data…
The Shape of Adversarial Influence: Characterizing LLM Latent Spaces with Persistent Homology
Aideen Fay, Inés García-Redondo, Qiquan Wang +2
Existing interpretability methods for Large Language Models (LLMs) predominantly capture linear directions or isolated features. This overlooks the high-dimensional, relational, an…
Metric Graph Kernels via the Tropical Torelli Map
Yueqi Cao, Anthea Monod
We introduce the first graph kernels for metric graphs via tropical algebraic geometry. In contrast to conventional graph kernels based on graph combinatorics such as nodes, edges,…
On the Limitations of Fractal Dimension as a Measure of Generalization
Charlie B. Tan, Inés García-Redondo, Qiquan Wang +2
Bounding and predicting the generalization gap of overparameterized neural networks remains a central open problem in theoretical machine learning. There is a recent and growing bo…
A Computational Tropical Geometry Framework for Neural Networks
Paul Lezeau, Thomas Walker, Yueqi Cao +2
We propose a computational tropical geometry framework for the symbolic analysis of neural networks with tropical activations. The number of linear regions of a neural network has…