29 citations · 91 across the 43 of their papers we have counts for
6 papers · 1 filter
The Surprising Expressivity of a Single Neuron with Autoregressive Chain-of-Thought
Roey Magen, Nirmit Joshi, Gal Vardi
The success of autoregressive language models trained with Chain-of-Thought~(CoT) supervision raises a fundamental question: How simple can a next-token predictor be while still en…
Observation of A Solar Like Magnetic Reconnection Event in an AGN Corona with XRISM
Gal Vardi, Ehud Behar, Liyi Gu +14
The X-ray source in AGN is commonly referred to as the corona by analogy to stellar coronae. The similarities between the two suggest that the heating mechanism of AGN coronae is m…
Learning to Think from Multiple Thinkers
Nirmit Joshi, Roey Magen, Nathan Srebro +2
We study learning with Chain-of-Thought (CoT) supervision from multiple thinkers, all of whom provide correct but possibly systematically different solutions, e.g., step-by-step so…
Positive Distribution Shift as a Framework for Understanding Tractable Learning
Marko Medvedev, Idan Attias, Elisabetta Cornacchia +3
We study a setting where the goal is to learn a target function f(x) with respect to a target distribution D(x), but training is done on i.i.d. samples from a different training di…
The Implicit Bias of Adam and Muon on Smooth Homogeneous Neural Networks
Eitan Gronich, Gal Vardi
We study the implicit bias of momentum-based optimizers on smooth homogeneous models. We show that \textit{momentum steepest descent} algorithms like Muon (spectral norm), Momentum…
To Grok Grokking: Provable Grokking in Ridge Regression
Mingyue Xu, Gal Vardi, Itay Safran
We study grokking, the onset of generalization long after overfitting, in a classical ridge regression setting. We prove end-to-end grokking results for learning over-parameterized…