3 papers
cs.LG2025
From superposition to sparse codes: interpretable representations in neural networks
David Klindt, Charles O'Neill, Patrik Reizinger +2
Understanding how information is represented in neural networks is a fundamental challenge in both neuroscience and artificial intelligence. Despite their nonlinear architectures,…
cs.LG2025
Compute Optimal Inference and Provable Amortisation Gap in Sparse Autoencoders
Charles O'Neill, Alim Gumran, David Klindt
A recent line of work has shown promise in using sparse autoencoders (SAEs) to uncover interpretable features in neural network representations. However, the simple linear-nonlinea…
cs.LG2025
Self-Attention as a Parametric Endofunctor: A Categorical Framework for Transformer Architectures
Charles O'Neill
Self-attention mechanisms have revolutionised deep learning architectures, yet their core mathematical structures remain incompletely understood. In this work, we develop a categor…