11 papers
Provable Learning of Random Hierarchy Models and Hierarchical Shallow-to-Deep Chaining
Yunwei Ren, Yatin Dandi, Florent Krzakala +1
The empirical success of deep learning is often attributed to deep networks' ability to exploit hierarchical structure in data, constructing increasingly complex features across la…
Scaling Laws from Sequential Feature Recovery: A Solvable Hierarchical Model
Arie Wortsman-Zurich, Hugo Tabanelli, Yatin Dandi +2
We propose a simple mechanism by which scaling laws emerge from feature learning in multi-layer networks. We study a high-dimensional hierarchical target that is a globally high-de…
Rigorous Asymptotics for First-Order Algorithms Through the Dynamical Cavity Method
Yatin Dandi, David Gamarnik, Francisco Pernice +1
Dynamical Mean Field Theory (DMFT) provides an asymptotic description of the dynamics of macroscopic observables in certain disordered systems. Originally pioneered in the context…
Deep Learning of Compositional Targets with Hierarchical Spectral Methods
Hugo Tabanelli, Yatin Dandi, Luca Pesce +1
Why depth yields a genuine computational advantage over shallow methods remains a central open question in learning theory. We study this question in a controlled high-dimensional…
Minimum Number of Monochromatic Subgraphs of a Random Graph
Yatin Dandi, David Gamarnik, Haodong Zhu
We consider the problem of minimizing the number of monochromatic subgraphs of a random graph, when each node of the host graph is assigned one of the two colors. Using a recently…
Asymptotics of Non-Convex Generalized Linear Models in High-Dimensions: A proof of the replica formula
Matteo Vilucchio, Yatin Dandi, Matéo Pirio Rossignol +2
The analytic characterization of the high-dimensional behavior of optimization for Generalized Linear Models (GLMs) with Gaussian data has been a central focus in statistics and pr…