1 citations · 1 across the 11 of their papers we have counts for
13 papers
Symmetry-Aware Feature Learning: A Polynomial Separation for Multi-Index Models
Jivan Waber, Vanessa Piccolo, Yatin Dandi +1
We establish a polynomial sample complexity separation between symmetry-aware and symmetry-agnostic feature learning. We study growing-rank multi-index models with high-dimensional…
Scaling Laws from Sequential Feature Recovery: A Solvable Hierarchical Model
Arie Wortsman-Zurich, Hugo Tabanelli, Yatin Dandi +2
We propose a simple mechanism by which scaling laws emerge from feature learning in multi-layer networks. We study a high-dimensional hierarchical target that is a globally high-de…
Rigorous Asymptotics for First-Order Algorithms Through the Dynamical Cavity Method
Yatin Dandi, David Gamarnik, Francisco Pernice +1
Dynamical Mean Field Theory (DMFT) provides an asymptotic description of the dynamics of macroscopic observables in certain disordered systems. Originally pioneered in the context…
Deep Learning of Compositional Targets with Hierarchical Spectral Methods
Hugo Tabanelli, Yatin Dandi, Luca Pesce +1
Why depth yields a genuine computational advantage over shallow methods remains a central open question in learning theory. We study this question in a controlled high-dimensional…
Minimum Number of Monochromatic Subgraphs of a Random Graph
Yatin Dandi, David Gamarnik, Haodong Zhu
We consider the problem of minimizing the number of monochromatic subgraphs of a random graph, when each node of the host graph is assigned one of the two colors. Using a recently…
Provable Learning of Random Hierarchy Models and Hierarchical Shallow-to-Deep Chaining
Yunwei Ren, Yatin Dandi, Florent Krzakala +1
The empirical success of deep learning is often attributed to deep networks' ability to exploit hierarchical structure in data, constructing increasingly complex features across la…
Sequential Dynamics in Ising Spin Glasses
Yatin Dandi, David Gamarnik, Francisco Pernice +1
We present the first exact asymptotic characterization of sequential dynamics for a broad class of local update algorithms on the Sherrington-Kirkpatrick (SK) model with Ising spin…
Fundamental limits of learning in sequence multi-index models and deep attention networks: High-dimensional asymptotics and sharp thresholds
Emanuele Troiani, Hugo Cui, Yatin Dandi +2
In this manuscript, we study the learning of deep attention neural networks, defined as the composition of multiple self-attention layers, with tied and low-rank weights. We first…
Asymptotics of Non-Convex Generalized Linear Models in High-Dimensions: A proof of the replica formula
Matteo Vilucchio, Yatin Dandi, Matéo Pirio Rossignol +2
The analytic characterization of the high-dimensional behavior of optimization for Generalized Linear Models (GLMs) with Gaussian data has been a central focus in statistics and pr…
The Computational Advantage of Depth: Learning High-Dimensional Hierarchical Functions with Gradient Descent
Yatin Dandi, Luca Pesce, Lenka Zdeborová +1
Understanding the advantages of deep neural networks trained by gradient descent (GD) compared to shallow models remains an open theoretical challenge. In this paper, we introduce…
Optimal Spectral Transitions in High-Dimensional Multi-Index Models
Leonardo Defilippis, Yatin Dandi, Pierre Mergny +2
We consider the problem of how many samples from a Gaussian multi-index model are required to weakly reconstruct the relevant index subspace. Despite its increasing popularity as a…
A Random Matrix Theory Perspective on the Spectrum of Learned Features and Asymptotic Generalization Capabilities
Yatin Dandi, Luca Pesce, Hugo Cui +3
A key property of neural networks is their capacity of adapting to data during training. Yet, our current mathematical understanding of feature learning and its relationship to gen…
Online Learning and Information Exponents: On The Importance of Batch size, and Time/Complexity Tradeoffs
Luca Arnaboldi, Yatin Dandi, Florent Krzakala +3
We study the impact of the batch size on the iteration time of training two-layer neural networks with one-pass stochastic gradient descent (SGD) on multi-index target fu…