8 citations · 10 across the 15 of their papers we have counts for
7 papers · 1 filter
Rank-1 LoRAs Encode Interpretable Reasoning Signals
Jake Ward, Paul Riechers, Adam Shai
Reasoning models leverage inference-time compute to significantly enhance the performance of language models on difficult logical tasks, and have become a dominating paradigm in fr…
Identifiability and minimality bounds of quantum and post-quantum models of classical stochastic processes
Paul M. Riechers, Thomas J. Elliott
To make sense of the world around us, we develop models, constructed to enable us to replicate, describe, and explain the behaviours we see. Focusing on the broad case of sequences…
Neural networks leverage nominally quantum and post-quantum representations
Paul M. Riechers, Thomas J. Elliott, Adam S. Shai
We show that deep neural networks, including transformers and RNNs, pretrained as usual on next-token prediction, intrinsically discover and represent beliefs over 'quantum' and 'p…
Next-token pretraining implies in-context learning
Paul M. Riechers, Henry R. Bigelow, Eric A. Alt +1
We argue that in-context learning (ICL) predictably arises from standard self-supervised next-token pretraining, rather than being an exotic emergent property. This work establishe…
Energetic advantages for quantum agents in online execution of complex strategies
Jayne Thompson, Paul M. Riechers, Andrew J. P. Garner +2
Agents often execute complex strategies -- adapting their response to each input stimulus depending on past observations and actions. Here, we derive the minimal energetic cost for…
Constrained belief updates explain geometric structures in transformer representations
Mateusz Piotrowski, Paul M. Riechers, Daniel Filan +1
What computational structures emerge in transformers trained on next-token prediction? In this work, we provide evidence that transformers implement constrained Bayesian belief upd…