1 citations · 2 across the 5 of their papers we have counts for
3 papers · 1 filter
Neural networks leverage nominally quantum and post-quantum representations
Paul M. Riechers, Thomas J. Elliott, Adam S. Shai
We show that deep neural networks, including transformers and RNNs, pretrained as usual on next-token prediction, intrinsically discover and represent beliefs over 'quantum' and 'p…
Next-token pretraining implies in-context learning
Paul M. Riechers, Henry R. Bigelow, Eric A. Alt +1
We argue that in-context learning (ICL) predictably arises from standard self-supervised next-token pretraining, rather than being an exotic emergent property. This work establishe…
Constrained belief updates explain geometric structures in transformer representations
Mateusz Piotrowski, Paul M. Riechers, Daniel Filan +1
What computational structures emerge in transformers trained on next-token prediction? In this work, we provide evidence that transformers implement constrained Bayesian belief upd…