82 citations · 246 across the 15 of their papers we have counts for
15 papers
There Will Be a Scientific Theory of Deep Learning
Jamie Simon, Daniel Kunin, Alexander Atanasov +11
In this paper, we make the case that a scientific theory of deep learning is emerging. By this we mean a theory which characterizes important properties and statistics of the train…
Two-Point Deterministic Equivalence for Stochastic Gradient Dynamics in Linear Models
Alexander Atanasov, Blake Bordelon, Jacob A. Zavatone-Veth +2
We derive a novel deterministic equivalence for the two-point function of a random matrix resolvent. Using this result, we give a unified derivation of the performance of a wide va…
The Optimization Landscape of SGD Across the Feature Learning Strength
Alexander Atanasov, Alexandru Meterez, James B. Simon +1
We consider neural networks (NNs) where the final layer is down-scaled by a fixed hyperparameter . Recent work has identified as controlling the strength of feature learning…
How Feature Learning Can Improve Neural Scaling Laws
Blake Bordelon, Alexander Atanasov, Cengiz Pehlevan
We develop a solvable model of neural scaling laws beyond the kernel limit. Theoretical analysis of this model shows how performance scales with model size, training time, and the…
Risk and cross validation in ridge regression with correlated samples
Alexander Atanasov, Jacob A. Zavatone-Veth, Cengiz Pehlevan
Recent years have seen substantial advances in our understanding of high-dimensional ridge regression, but existing theories assume that training examples are independent. By lever…
Scaling and renormalization in high-dimensional regression
Alexander Atanasov, Jacob A. Zavatone-Veth, Cengiz Pehlevan
From benign overfitting in overparameterized models to rich power-law scalings in performance, simple ridge regression displays surprising behaviors sometimes thought to be limited…
A Dynamical Model of Neural Scaling Laws
Blake Bordelon, Alexander Atanasov, Cengiz Pehlevan
On a variety of tasks, the performance of neural networks predictably improves with training time, dataset size and model size across many orders of magnitude. This phenomenon is k…
Feature-Learning Networks Are Consistent Across Widths At Realistic Scales
Nikhil Vyas, Alexander Atanasov, Blake Bordelon +3
We study the effect of width on the dynamics of feature-learning neural networks across a variety of architectures and datasets. Early in training, wide neural networks trained on…
The Onset of Variance-Limited Behavior for Networks in the Lazy and Rich Regimes
Alexander Atanasov, Blake Bordelon, Sabarish Sainathan +1
For small training set sizes , the generalization error of wide neural networks is well-approximated by the error of an infinite width neural network (NN), either in the kernel…
Precision Bootstrap for the Super-Ising Model
Alexander Atanasov, Aaron Hillman, David Poland +2
In this note we report an improved determination of the scaling dimensions and OPE coefficients of the minimal supersymmetric extension of the 3d Ising model using the conformal bo…
Neural Networks as Kernel Learners: The Silent Alignment Effect
Alexander Atanasov, Blake Bordelon, Cengiz Pehlevan
Neural networks in the lazy training regime converge to kernel machines. Can neural networks in the rich feature learning regime learn a kernel machine with a data-dependent kernel…
Conformal Block Expansion in Celestial CFT
Alexander Atanasov, Walker Melton, Ana-Maria Raclariu +1
The 4D 4-point scattering amplitude of massless scalars via a massive exchange is expressed in a basis of conformal primary particle wavefunctions. This celestial amplitude is expa…
Scattering and the Celestial Torus
Alexander Atanasov, Adam Ball, Walker Melton +2
Analytic continuation from Minkowski space to split signature spacetime has proven to be a powerful tool for the study of scattering amplitudes. Here we show that, under th…
Bootstrapping the Minimal 3D SCFT
Alexander Atanasov, Aaron Hillman, David Poland
We study the conformal bootstrap constraints for 3D conformal field theories with a or parity symmetry, assuming a single relevant scalar operator that is invari…
Sparse Grid Discretizations based on a Discontinuous Galerkin Method
Alexander B. Atanasov, Erik Schnetter
We examine and extend Sparse Grids as a discretization method for partial differential equations (PDEs). Solving a PDE in dimensions has a cost that grows as with comm…