1.2k citations · 2.2k across the 46 of their papers we have counts for
5 papers · 1 filter
RLVR landscapes for iterated multiplications can be benign: Insights from spin-glass theory
Noa Rubin, Zohar Ringel
Despite the importance of reinforcement learning with verifiable rewards (RLVR), the extent to which it can learn new reasoning capabilities remains debated. Here we study the opti…
Phase Transitions in Attention: A Bayesian Theory of Copy Head Emergence
Itay Lavie, Kirsten Fischer, Andrey Lekov +3
Attention is the key mechanism underlying in-context learning in transformers, and attention patterns have been observed empirically to emerge abruptly during training. We present…
Improving CFT Operators Using Machine Learning
Lior Oppenheim, Snir Gazit, Zohar Ringel
Finite-size effects limit the accuracy with which conformal data can be extracted from lattice simulations of critical systems. While action improvement suppresses some corrections…
Algorithmic Task Capture, Computational Complexity, and Inductive Bias of Infinite Transformers
Orit Davidovich, Zohar Ringel
We formally define algorithmic capture of combinatorial tasks as the ability of a transformer to extrapolate to arbitrary task sizes with controllable error and logarithmic sample…
Lecture notes: From Gaussian processes to feature learning
Moritz Helias, Javed Lindner, Lars Schutzeichel +1
These lecture notes develop the theory of learning in deep and recurrent neuronal networks from the point of view of Bayesian inference. The aim is to enable the reader to understa…