9 citations · 15 across the 15 of their papers we have counts for
11 papers · 1 filter
Relating Piecewise Linear Kolmogorov Arnold Networks to ReLU Networks
Nandi Schoots, Mattia Jacopo Villani, Niels uit de Bos
Kolmogorov-Arnold Networks are a new family of neural network architectures which holds promise for overcoming the curse of dimensionality and has interpretability benefits (arXiv:…
Studying Cross-cluster Modularity in Neural Networks
Satvik Golechha, Maheep Chaudhary, Joan Velja +2
An approach to improve neural network interpretability is via clusterability, i.e., splitting a model into disjoint clusters that can be studied independently. We define a measure…
Open Problems in Mechanistic Interpretability
Lee Sharkey, Bilal Chughtai, Joshua Batson +26
Mechanistic interpretability aims to understand the computational mechanisms underlying neural networks' capabilities in order to accomplish concrete scientific and engineering goa…
The Propensity for Density in Feed-forward Models
Nandi Schoots, Alex Jackson, Ali Kholmovaia +2
Does the process of training a neural network to solve a task tend to use all of the available weights even when the task could be solved with fewer weights? To address this questi…
Training Neural Networks for Modularity aids Interpretability
Satvik Golechha, Dylan Cope, Nandi Schoots
An approach to improve network interpretability is via clusterability, i.e., splitting a model into disjoint clusters that can be studied independently. We find pretrained models t…
Extending Activation Steering to Broad Skills and Multiple Behaviours
Teun van der Weij, Massimo Poesio, Nandi Schoots
Current large language models have dangerous capabilities, which are likely to become more problematic in the future. Activation steering techniques can be used to reduce risks fro…
Dissecting Language Models: Machine Unlearning via Selective Pruning
Nicholas Pochinkov, Nandi Schoots
Understanding and shaping the behaviour of Large Language Models (LLMs) is increasingly important as applications become more powerful and more frequently adopted. This paper intro…
Comparing Optimization Targets for Contrast-Consistent Search
Hugo Fry, Seamus Fallows, Ian Fan +2
We investigate the optimization target of Contrast-Consistent Search (CCS), which aims to recover the internal representations of truth of a large language model. We present a new…
Any Deep ReLU Network is Shallow
Mattia Jacopo Villani, Nandi Schoots
We constructively prove that every deep ReLU network can be rewritten as a functionally identical three-layer network with weights valued in the extended reals. Based on this proof…
Low-Entropy Latent Variables Hurt Out-of-Distribution Performance
Nandi Schoots, Dylan Cope
We study the relationship between the entropy of intermediate representations and a model's robustness to distributional shift. We train models consisting of two feed-forward netwo…
Learning to Communicate with Strangers via Channel Randomisation Methods
Dylan Cope, Nandi Schoots
We introduce two methods for improving the performance of agents meeting for the first time to accomplish a communicative task. The methods are: (1) `message mutation' during the g…