38 citations · 80 across the 10 of their papers we have counts for
4 papers · 1 filter
Beyond Tokens in Language Models: Interpreting Activations through Text Genre Chunks
Éloïse Benito-Rodriguez, Einar Urdshals, Jasmina Nasufi +1
Understanding Large Language Models (LLMs) is key to ensure their safe and beneficial deployment. This task is complicated by the difficulty of interpretability of LLM structures,…
Compressibility Measures Complexity: Minimum Description Length Meets Singular Learning Theory
Einar Urdshals, Edmund Lau, Jesse Hoogland +2
We study neural network compressibility by using singular learning theory to extend the minimum description length (MDL) principle to singular models like neural networks. Through…
Electronic structure of liquid xenon in the context of light dark matter direct detection
Riccardo Catena, Luca Marin, Marek Matas +2
We present a description of the electronic structure of xenon within the density-functional theory formalism with the goal of accurately modeling dark-matter-induced ionisation in…
Structure Development in List-Sorting Transformers
Einar Urdshals, Jasmina Urdshals
We study how a one-layer attention-only transformer develops relevant structures while learning to sort lists of numbers. At the end of training, the model organizes its attention…