9 citations · 12 across the 14 of their papers we have counts for
20 papers
Interpreting Language Model Parameters
Lucius Bushnaq, Dan Braun, Oliver Clive-Griffin +5
Mechanistic interpretability aims to uncover how neural networks use their parameters to implement their impressive neural algorithms. Although previous work has uncovered substant…
Language Model Activations Inhabit Privileged Error-Correcting Basins
Matthew Finlayson, Francisco Pernice, Eric Todd +16
Language models exhibit remarkable robustness, continuing to produce coherent text even when their activations are perturbed by interventions like linear steering. We hypothesize t…
Functional Degeneracy in Neural Networks: Measurement and Pruning
Maria Matveev, Pascal Esser, Ayush Bharadwaj +2
A central question in modern machine learning is how much a trained model can be compressed without changing its behavior, to reduce the memory, compute and energy required to depl…
Structuring Sparsity: Block-Sparse Featurizers Capture Visual Concept Manifolds
Thomas Fel, Matthew Kowal, Mozes Jacobs +22
What is the geometry of a visual percept? The most widely used protocols for decomposing neural network representations into interpretable parts treat concepts as isolated directio…
From Memorization to Reasoning in the Spectrum of Loss Curvature
Jack Merullo, Srihita Vatsavaya, Lucius Bushnaq +1
We characterize how memorization is represented in transformer models and show that it can be disentangled in the weights of both language models (LMs) and vision transformers (ViT…
Stochastic Parameter Decomposition
Lucius Bushnaq, Dan Braun, Lee Sharkey
A key step in reverse engineering neural networks is to decompose them into simpler parts that can be studied in relative isolation. Linear parameter decomposition -- a framework t…
Identifying Sparsely Active Circuits Through Local Loss Landscape Decomposition
Brianna Chrisman, Lucius Bushnaq, Lee Sharkey
Much of mechanistic interpretability has focused on understanding the activation spaces of large neural networks. However, activation space-based approaches reveal little about the…
Interpretability in Parameter Space: Minimizing Mechanistic Description Length with Attribution-based Parameter Decomposition
Dan Braun, Lucius Bushnaq, Stefan Heimersheim +2
Mechanistic interpretability aims to understand the internal mechanisms learned by neural networks. Despite recent progress toward this goal, it remains unclear how best to decompo…
Open Problems in Mechanistic Interpretability
Lee Sharkey, Bilal Chughtai, Joshua Batson +26
Mechanistic interpretability aims to understand the computational mechanisms underlying neural networks' capabilities in order to accomplish concrete scientific and engineering goa…
Towards evaluations-based safety cases for AI scheming
Mikita Balesni, Marius Hobbhahn, David Lindner +13
We sketch how developers of frontier AI systems could construct a structured rationale -- a 'safety case' -- that an AI system is unlikely to cause catastrophic outcomes through sc…
The Local Interaction Basis: Identifying Computationally-Relevant and Sparsely Interacting Features in Neural Networks
Lucius Bushnaq, Stefan Heimersheim, Nicholas Goldowsky-Dill +7
Mechanistic interpretability aims to understand the behavior of neural networks by reverse-engineering their internal computations. However, current methods struggle to find clear…
Using Degeneracy in the Loss Landscape for Mechanistic Interpretability
Lucius Bushnaq, Jake Mendel, Stefan Heimersheim +5
Mechanistic Interpretability aims to reverse engineer the algorithms implemented by neural networks by studying their weights and activations. An obstacle to reverse engineering ne…
Strange and charm contributions to the HVP from C* boundary conditions
Anian Altherr, Lucius Bushnaq, Isabel Campos +12
We present preliminary results for the determination of the leading strange and charm quark-connected contributions to the hadronic vacuum polarization contribution to the muon's g…
Hadronic vacuum polarization with C* boundary conditions
Anian Altherr, Roman Gruber, Lucius Bushnaq +12
We present a progress report on the calculation of the connected hadronic contribution to the muon g-2 with C* boundary conditions. For that purpose we use a QCD gauge ensemble wit…
Tuning of QCD+QED simulations with C boundary conditions
Anian Altherr, Lucius Bushnaq, Isabel Campos +10
We give an update on the ongoing effort of the RC collaboration to generate fully dynamical QCD+QED ensembles with C boundary conditions using the openQD co…
QCD+QED simulations with C boundary conditions
Anian Altherr, Lucius Bushnaq, Isabel Campos +10
We give an update on the ongoing effort of the RC collaboration to generate fully dynamical QCD+QED configurations with C boundary conditions using the openQ$^\star…
First results on QCD+QED with C* boundary conditions
Lucius Bushnaq, Isabel Campos, Marco Catillo +7
Accounting for isospin-breaking corrections is critical for achieving subpercent precision in lattice computations of hadronic observables. A way to include QED and strong-isospin-…
Implementing noise reduction techniques into theOpenQ*D package
Lucius Bushnaq, Isabel Campos, Marco Catillo +7
We present the results of testing a new technique for stochastic noise reduction in the calculation of propagators by implementing it in OpenQ*D for two ensembles with O(a) improve…
Baryon masses from full QCD+QED simulations
Lucius Bushnaq, Isabel Campos, Marco Catillo +7
In these proceedings we present preliminary results for the masses of the proton, neutron and baryons obtained from QCD+QED lattice simulations performed with four dynamical…
An update on QCD+QED simulations with C* boundary conditions
Lucius Bushnaq, Isabel Campos, Marco Catillo +7
We present two novelties in our analysis of fully dynamical QCD+QED ensembles with C* boundary conditions. The first one is the explicit computation of the sign of the Pfaffian. We…