4 papers
Predicting Linear Relational Decodability Without the Relation
Jonas Rohweder, Adrian Mülthaler, Faridoun Mehri +3
Linear relational embeddings (LREs) approximate a language model's subject-to-object mapping with an affine function. Their faithfulness varies enormously across relations, from ne…
GLUScope: A Tool for Analyzing GLU Neurons in Transformer Language Models
Sebastian Gerstner, Hinrich Schütze, Leonor Veloso
We present **GLUScope**, an open-source resource for analyzing neurons in Transformer-based language models (LMs), intended for interpretability researchers. We focus on the now-st…
BlackboxNLP-2025 MIB Shared Task: Exploring Ensemble Strategies for Circuit Localization Methods
Philipp Mondorf, Mingyang Wang, Sebastian Gerstner +6
The Circuit Localization track of the Mechanistic Interpretability Benchmark (MIB) evaluates methods for localizing circuits within large language models (LLMs), i.e., subnetworks…
Understanding Gated Neurons in Transformers from Their Input-Output Functionality
Sebastian Gerstner, Hinrich Schütze
Interpretability researchers have attempted to understand MLP neurons of language models based on both the contexts in which they activate and their output weight vectors. They hav…