1 citations · 1 across the 5 of their papers we have counts for
6 papers · 1 filter
In-Context Learning Before Specialisation: Theory of Learning in a Solvable Topic Model
Francesco Cagnetta, Casper L. Christensen
Sequence models can generalise in fundamentally distinct ways: extrapolating structures seen in training, or using their prompt to adapt to novel tasks. The prevailing strategy det…
LLMs program using Reusable and Composable Abstract Representations
Casper L. Christensen, Kyle J. Ray, Dmitry Manning-Coe +2
When writing code, the same computation can be expressed in different programming languages and through different implementation strategies. We show that the activations of LLMs ca…
Transformers learn factored representations
Adam Shai, Loren Amdahl-Culleton, Casper L. Christensen +6
Transformers pretrained via next token prediction learn to factor their world into parts, representing these factors in orthogonal subspaces of the residual stream. We formalize tw…
Decomposition of Small Transformer Models
Casper L. Christensen, Logan Riggs
Recent work in mechanistic interpretability has shown that decomposing models in parameter space may yield clean handles for analysis and intervention. Previous methods have demons…
Bi-Axial Transformers: Addressing the Increasing Complexity of EHR Classification
Rachael DeVries, Casper Christensen, Marie Lisandra Zepeda Mendoza +1
Electronic Health Records (EHRs), the digital representation of a patient's medical history, are a valuable resource for epidemiological and clinical research. They are also becomi…
Normalized AOPC: Fixing Misleading Faithfulness Metrics for Feature Attribution Explainability
Joakim Edin, Andreas Geert Motzfeldt, Casper L. Christensen +3
Deep neural network predictions are notoriously difficult to interpret. Feature attribution methods aim to explain these predictions by identifying the contribution of each input f…