91 citations · 257 across the 9 of their papers we have counts for
10 papers
Six Lectures on Linearized Neural Networks
Theodor Misiakiewicz, Andrea Montanari
In these six lectures, we examine what can be learnt about the behavior of multi-layer neural networks from the analysis of linear models. We first recall the correspondence betwee…
Learning with convolution and pooling operations in kernel methods
Theodor Misiakiewicz, Song Mei
Recent empirical work has shown that hierarchical convolutional kernels inspired by convolutional neural networks (CNNs) significantly improve the performance of kernel methods in…
Minimum complexity interpolation in random features models
Michael Celentano, Theodor Misiakiewicz, Andrea Montanari
Despite their many appealing properties, kernel methods are heavily affected by the curse of dimensionality. For instance, in the case of inner product kernels in , t…
Learning with invariances in random features and kernel models
Song Mei, Theodor Misiakiewicz, Andrea Montanari
A number of machine learning tasks entail a high degree of invariance: the data distribution does not change if we act on the data with a certain group of transformations. For inst…
Generalization error of random features and kernel methods: hypercontractivity and kernel matrix concentration
Song Mei, Theodor Misiakiewicz, Andrea Montanari
Consider the classical supervised learning problem: we are given data , , with a response and a covariate…
When Do Neural Networks Outperform Kernel Methods?
Behrooz Ghorbani, Song Mei, Theodor Misiakiewicz +1
For a certain scaling of the initialization of stochastic gradient descent (SGD), wide neural networks (NN) have been shown to be well approximated by reproducing kernel Hilbert sp…
Limitations of Lazy Training of Two-layers Neural Networks
Behrooz Ghorbani, Song Mei, Theodor Misiakiewicz +1
We study the supervised learning problem under either of the following two models: (1) Feature vectors are -dimensional Gaussians and responses are $y_i = f_…
Linearized two-layers neural networks in high dimension
Behrooz Ghorbani, Song Mei, Theodor Misiakiewicz +1
We consider the problem of learning an unknown function on the -dimensional sphere with respect to the square loss, given i.i.d. samples $\{(y_i,{\boldsymbol x}_i)\}…
Mean-field theory of two-layers neural networks: dimension-free bounds and kernel limit
Song Mei, Theodor Misiakiewicz, Andrea Montanari
We consider learning two layer neural networks using stochastic gradient descent. The mean-field description of this learning dynamics approximates the evolution of the network wei…
Solving SDPs for synchronization and MaxCut problems via the Grothendieck inequality
Song Mei, Theodor Misiakiewicz, Andrea Montanari +1
A number of statistical estimation problems can be addressed by semidefinite programs (SDP). While SDPs are solvable in polynomial time using interior point methods, in practice ge…