3 citations · 3 across the 9 of their papers we have counts for
10 papers
From Mechanistic to Compositional Interpretability
Ward Gauderis, Thomas Dooms, Steven T. Homer +2
Mechanistic interpretability aims to explain neural model behaviour by reverse-engineering learned computational structure into human-understandable components. Without a formal fr…
When Are Two Networks the Same? Tensor Similarity for Mechanistic Interpretability
ML Nissen Gonzalez, Melwina Albuquerque, Laurence Wroe +3
Mechanistic interpretability aims to break models into meaningful parts; verifying that two such parts implement the same computation is a prerequisite. Existing similarity measure…
Bilinear autoencoders find interpretable manifolds
Thomas Dooms, Ward Gauderis, Geraint Wiggins +1
Sparse autoencoders have become a standard tool for uncovering interpretable latent representations in neural networks. Yet salient concepts often span manifolds that current linea…
Finding Manifolds With Bilinear Autoencoders
Thomas Dooms, Ward Gauderis
Sparse autoencoders are a standard tool for uncovering interpretable latent representations in neural networks. Yet, their interpretation depends on the inputs, making their isolat…
Parameterized Synthetic Text Generation with SimpleStories
Lennart Finke, Chandan Sreedhara, Thomas Dooms +6
We present SimpleStories, a large synthetic story dataset in simple language, consisting of 2 million samples each in English and Japanese. Through parameterizing prompts at multip…
Compositionality Unlocks Deep Interpretable Models
Thomas Dooms, Ward Gauderis, Geraint A. Wiggins +1
We propose -net, an intrinsically interpretable architecture combining the compositional multilinear structure of tensor networks with the expressivity and efficiency of deep ne…
Tokenized SAEs: Disentangling SAE Reconstructions
Thomas Dooms, Daniel Wilhelm
Sparse auto-encoders (SAEs) have become a prevalent tool for interpreting language models' inner workings. However, it is unknown how tightly SAE features correspond to computation…
Bilinear MLPs enable weight-based mechanistic interpretability
Michael T. Pearce, Thomas Dooms, Alice Rigg +2
A mechanistic understanding of how MLPs do computation in deep neural networks remains elusive. Current interpretability work can extract features from hidden activations over an i…
Weight-based Decomposition: A Case for Bilinear MLPs
Michael T. Pearce, Thomas Dooms, Alice Rigg
Gated Linear Units (GLUs) have become a common building block in modern foundation models. Bilinear layers drop the non-linearity in the "gate" but still have comparable performanc…
The Trifecta: Three simple techniques for training deeper Forward-Forward networks
Thomas Dooms, Ing Jyh Tsang, Jose Oramas
Modern machine learning models are able to outperform humans on a variety of non-trivial tasks. However, as the complexity of the models increases, they consume significant amounts…