9 citations · 15 across the 15 of their papers we have counts for
15 papers
Punctuation and Predicates in Language Models
Sonakshi Chauhan, Maheep Chaudhary, Koby Choy +2
In this paper we explore where information is collected and how it is propagated throughout layers in large language models (LLMs). We begin by examining the surprising computation…
Relating Piecewise Linear Kolmogorov Arnold Networks to ReLU Networks
Nandi Schoots, Mattia Jacopo Villani, Niels uit de Bos
Kolmogorov-Arnold Networks are a new family of neural network architectures which holds promise for overcoming the curse of dimensionality and has interpretability benefits (arXiv:…
Studying Cross-cluster Modularity in Neural Networks
Satvik Golechha, Maheep Chaudhary, Joan Velja +2
An approach to improve neural network interpretability is via clusterability, i.e., splitting a model into disjoint clusters that can be studied independently. We define a measure…
Open Problems in Mechanistic Interpretability
Lee Sharkey, Bilal Chughtai, Joshua Batson +26
Mechanistic interpretability aims to understand the computational mechanisms underlying neural networks' capabilities in order to accomplish concrete scientific and engineering goa…
The Propensity for Density in Feed-forward Models
Nandi Schoots, Alex Jackson, Ali Kholmovaia +2
Does the process of training a neural network to solve a task tend to use all of the available weights even when the task could be solved with fewer weights? To address this questi…
Hidden in Plain Text: Emergence & Mitigation of Steganographic Collusion in LLMs
Yohan Mathew, Ollie Matthews, Robert McCarthy +4
The rapid proliferation of frontier model agents promises significant societal advances but also raises concerns about systemic risks arising from unsafe interactions. Collusion to…
Training Neural Networks for Modularity aids Interpretability
Satvik Golechha, Dylan Cope, Nandi Schoots
An approach to improve network interpretability is via clusterability, i.e., splitting a model into disjoint clusters that can be studied independently. We find pretrained models t…
Extending Activation Steering to Broad Skills and Multiple Behaviours
Teun van der Weij, Massimo Poesio, Nandi Schoots
Current large language models have dangerous capabilities, which are likely to become more problematic in the future. Activation steering techniques can be used to reduce risks fro…
Dissecting Language Models: Machine Unlearning via Selective Pruning
Nicholas Pochinkov, Nandi Schoots
Understanding and shaping the behaviour of Large Language Models (LLMs) is increasingly important as applications become more powerful and more frequently adopted. This paper intro…
Improving Activation Steering in Language Models with Mean-Centring
Ole Jorgensen, Dylan Cope, Nandi Schoots +1
Recent work in activation steering has demonstrated the potential to better control the outputs of Large Language Models (LLMs), but it involves finding steering vectors. This is d…
Comparing Optimization Targets for Contrast-Consistent Search
Hugo Fry, Seamus Fallows, Ian Fan +2
We investigate the optimization target of Contrast-Consistent Search (CCS), which aims to recover the internal representations of truth of a large language model. We present a new…
Any Deep ReLU Network is Shallow
Mattia Jacopo Villani, Nandi Schoots
We constructively prove that every deep ReLU network can be rewritten as a functionally identical three-layer network with weights valued in the extended reals. Based on this proof…
Low-Entropy Latent Variables Hurt Out-of-Distribution Performance
Nandi Schoots, Dylan Cope
We study the relationship between the entropy of intermediate representations and a model's robustness to distributional shift. We train models consisting of two feed-forward netwo…
A theory of representation learning gives a deep generalisation of kernel methods
Adam X. Yang, Maxime Robeyns, Edward Milsom +3
The successes of modern deep machine learning methods are founded on their ability to transform inputs across multiple layers to build good high-level representations. It is theref…
Learning to Communicate with Strangers via Channel Randomisation Methods
Dylan Cope, Nandi Schoots
We introduce two methods for improving the performance of agents meeting for the first time to accomplish a communicative task. The methods are: (1) `message mutation' during the g…