70 citations · 85 across the 19 of their papers we have counts for
19 papers
Exact Dynamics of Multi-class Stochastic Gradient Descent
Elizabeth Collins-Woodfin, Inbar Seroussi
We develop a framework for analyzing the learning dynamics of high-dimensional problems trained using one-pass stochastic gradient descent (SGD) with data from multiple anisotropic…
Minimax Rates for Learning Pairwise Interactions in Attention-Style Models
Shai Zucker, Xiong Wang, Fei Lu +1
We study the convergence rate of learning pairwise interactions in single-layer attention-style models, where tokens interact through a weight matrix and a nonlinear activation fun…
A Geometric Unification of Generative AI with Manifold-Probabilistic Projection Models
Leah Bar, Liron Mor Yosef, Shai Zucker +3
Most models of generative AI for images assume that images are inherently low-dimensional objects embedded within a high-dimensional space. Additionally, it is often implicitly ass…
High-Dimensional Private Linear Regression with Optimal Rates
Simone Bombari, Jialei Luo, Inbar Seroussi +1
Differentially private (DP) linear regression has received significant attention in the recent theoretical literature, with several approaches proposed to improve error rates. Our…
Applications of Statistical Field Theory in Deep Learning
Zohar Ringel, Noa Rubin, Edo Mor +2
Deep learning algorithms have made incredible strides in the past decade, yet due to their complexity, the science of deep learning remains in its early stages. Being an experiment…
From Kernels to Features: A Multi-Scale Adaptive Theory of Feature Learning
Noa Rubin, Kirsten Fischer, Javed Lindner +5
Feature learning in neural networks is crucial for their expressive power and inductive biases, motivating various theoretical approaches. Some approaches describe network behavior…
The High Line: Exact Risk and Learning Rate Curves of Stochastic Adaptive Learning Rate Algorithms
Elizabeth Collins-Woodfin, Inbar Seroussi, Begoña García Malaxechebarría +3
We develop a framework for analyzing the training and learning rate dynamics on a large class of high-dimensional optimization problems, which we call the high line, trained using…
Optimal minimax rate of learning nonlocal interaction kernels
Xiong Wang, Inbar Seroussi, Fei Lu
Nonparametric estimation of nonlocal interaction kernels is crucial in various applications involving interacting particle systems. The inference challenge, situated at the nexus o…
Grokking as a First Order Phase Transition in Two Layer Networks
Noa Rubin, Inbar Seroussi, Zohar Ringel
A key property of deep neural networks (DNNs) is their ability to learn new features during training. This intriguing aspect of deep learning stands out most clearly in recently re…
Hitting the High-Dimensional Notes: An ODE for SGD learning dynamics on GLMs and multi-index models
Elizabeth Collins-Woodfin, Courtney Paquette, Elliot Paquette +1
We analyze the dynamics of streaming stochastic gradient descent (SGD) in the high-dimensional limit when applied to generalized linear models and multi-index models (e.g. logistic…
Speed Limits for Deep Learning
Inbar Seroussi, Alexander A. Alemi, Moritz Helias +1
State-of-the-art neural networks require extreme computational power to train. It is therefore natural to wonder whether they are optimally trained. Here we apply a recent advancem…
Spectral-Bias and Kernel-Task Alignment in Physically Informed Neural Networks
Inbar Seroussi, Asaf Miron, Zohar Ringel
Physically informed neural networks (PINNs) are a promising emerging method for solving differential equations. As in many other deep learning approaches, the choice of PINN design…
Separation of Scales and a Thermodynamic Description of Feature Learning in Some CNNs
Inbar Seroussi, Gadi Naveh, Zohar Ringel
Deep neural networks (DNNs) are powerful tools for compressing and distilling information. Their scale and complexity, often involving billions of inter-dependent parameters, rende…
Lower Bounds on the Generalization Error of Nonlinear Learning Models
Inbar Seroussi, Ofer Zeitouni
We study in this paper lower bounds for the generalization error of models derived from multi-layer neural networks, in the regime where the size of the layers is commensurate with…
Directed polymers on infinite graphs
Clement Cosco, Inbar Seroussi, Ofer Zeitouni
We study the directed polymer model for general graphs (beyond ) and random walks. We provide sufficient conditions for the existence or non-existence of a weak disord…
From Logistic Growth to Exponential Growth in a Population Dynamical Model
Inbar Seroussi, Nir Sochen
Dynamics among central sources (hubs) providing a resource and large number of components enjoying and contributing to this resource describes many real life situations. Modeling,…
On the use of multiple compartment epidemiological models to describe the dynamics of influenza in Europe
Inbar Seroussi, Nir Levy, Daniela Paolotti +2
We develop a multiple compartment Susceptible-Infected-Recovered (SIR) model to analyze the spread of several infectious diseases through different geographic areas. Additionally,…
Multi-Season Analysis Reveals the Spatial Structure of Disease Spread
Inbar Seroussi, Nir Levy, Elad Yom-Tov
Understanding the dynamics of infectious disease spread in a heterogeneous population is an important factor in designing control strategies. Here, we develop a novel tensor-driven…
Topological Superconducting Phases of Weakly Coupled Quantum Wires
Inbar Seroussi, Erez Berg, Yuval Oreg
An array of quantum wires is a natural starting point in realizing two-dimensional topological phases. We study a system of weakly coupled quantum wires with Rashba spin-orbit coup…