70 citations · 85 across the 19 of their papers we have counts for
6 papers · 1 filter
Exact Dynamics of Multi-class Stochastic Gradient Descent
Elizabeth Collins-Woodfin, Inbar Seroussi
We develop a framework for analyzing the learning dynamics of high-dimensional problems trained using one-pass stochastic gradient descent (SGD) with data from multiple anisotropic…
Minimax Rates for Learning Pairwise Interactions in Attention-Style Models
Shai Zucker, Xiong Wang, Fei Lu +1
We study the convergence rate of learning pairwise interactions in single-layer attention-style models, where tokens interact through a weight matrix and a nonlinear activation fun…
A Geometric Unification of Generative AI with Manifold-Probabilistic Projection Models
Leah Bar, Liron Mor Yosef, Shai Zucker +3
Most models of generative AI for images assume that images are inherently low-dimensional objects embedded within a high-dimensional space. Additionally, it is often implicitly ass…
High-Dimensional Private Linear Regression with Optimal Rates
Simone Bombari, Jialei Luo, Inbar Seroussi +1
Differentially private (DP) linear regression has received significant attention in the recent theoretical literature, with several approaches proposed to improve error rates. Our…
Applications of Statistical Field Theory in Deep Learning
Zohar Ringel, Noa Rubin, Edo Mor +2
Deep learning algorithms have made incredible strides in the past decade, yet due to their complexity, the science of deep learning remains in its early stages. Being an experiment…
From Kernels to Features: A Multi-Scale Adaptive Theory of Feature Learning
Noa Rubin, Kirsten Fischer, Javed Lindner +5
Feature learning in neural networks is crucial for their expressive power and inductive biases, motivating various theoretical approaches. Some approaches describe network behavior…