45 citations · 286 across the 32 of their papers we have counts for
35 papers
LLMs program using Reusable and Composable Abstract Representations
Casper L. Christensen, Kyle J. Ray, Dmitry Manning-Coe +2
When writing code, the same computation can be expressed in different programming languages and through different implementation strategies. We show that the activations of LLMs ca…
Large Language Models Develop Belief State Geometry In-Context
Daniel Balcells, Andrew Jun Lee, Chirag Rastogi +3
Large language models (LLMs) trained on next-token prediction exhibit remarkable in-context learning (ICL) abilities, yet the representations that support ICL remain poorly underst…
Time-ordered free energy in correlated quantum systems: An agentic approach
Ruo Cheng Huang, Isha Singh Le Xue, Yuxuan Qu +3
How much work can an agent extract from a temporal sequence of quantum states when it can only operate online under causal constraints---deciding which energy extraction method to…
Transformers learn factored representations
Adam Shai, Loren Amdahl-Culleton, Casper L. Christensen +6
Transformers pretrained via next token prediction learn to factor their world into parts, representing these factors in orthogonal subspaces of the residual stream. We formalize tw…
Rank-1 LoRAs Encode Interpretable Reasoning Signals
Jake Ward, Paul Riechers, Adam Shai
Reasoning models leverage inference-time compute to significantly enhance the performance of language models on difficult logical tasks, and have become a dominating paradigm in fr…
Identifiability and minimality bounds of quantum and post-quantum models of classical stochastic processes
Paul M. Riechers, Thomas J. Elliott
To make sense of the world around us, we develop models, constructed to enable us to replicate, describe, and explain the behaviours we see. Focusing on the broad case of sequences…