26 citations · 137 across the 41 of their papers we have counts for
10 papers · 1 filter
Fundamental Limits of Two-layer Autoencoders, and Achieving Them with Gradient Methods
Alexander Shevchenko, Kevin Kögler, Hamed Hassani +1
Autoencoders are a popular model in many branches of machine learning and lossy data compression. However, their fundamental limits, the performance of gradient methods and the fea…
Approximate Message Passing for Multi-Layer Estimation in Rotationally Invariant Models
Yizhou Xu, TianQi Hou, ShanSuo Liang +1
We consider the problem of reconstructing the signal and the hidden variables from observations coming from a multi-layer network with rotationally invariant weight matrices. The m…
Finite Sample Identification of Wide Shallow Neural Networks with Biases
Massimo Fornasier, Timo Klock, Marco Mondelli +1
Artificial neural networks are functions depending on a finite number of parameters typically encoded as weights and biases. The identification of the parameters of the network fro…
Precise Asymptotics for Spectral Methods in Mixed Generalized Linear Models
Yihan Zhang, Marco Mondelli, Ramji Venkataramanan
In a mixed generalized linear model, the goal is to learn multiple signals from unlabeled observations: each sample comes from exactly one signal, but it is not known which one. We…
Mean-field analysis for heavy ball methods: Dropout-stability, connectivity, and global convergence
Diyuan Wu, Vyacheslav Kungurtsev, Marco Mondelli
The stochastic heavy ball method (SHB), also known as stochastic gradient descent (SGD) with Polyak's momentum, is widely used in training neural networks. However, despite the rem…
Bayes-optimal limits in structured PCA, and how to reach them
Jean Barbier, Francesco Camilli, Marco Mondelli +1
How do statistical dependencies in measurement noise influence high-dimensional inference? To answer this, we study the paradigmatic spiked matrix model of principal components ana…
The price of ignorance: how much does it cost to forget noise structure in low-rank matrix estimation?
Jean Barbier, TianQi Hou, Marco Mondelli +1
We consider the problem of estimating a rank-1 signal corrupted by structured rotationally invariant noise, and address the following question: how well do inference algorithms per…
Memorization and Optimization in Deep Neural Networks with Minimum Over-parameterization
Simone Bombari, Mohammad Hossein Amani, Marco Mondelli
The Neural Tangent Kernel (NTK) has emerged as a powerful tool to provide memorization, optimization and generalization guarantees in deep neural networks. A line of work has studi…
Sharp asymptotics on the compression of two-layer neural networks
Mohammad Hossein Amani, Simone Bombari, Marco Mondelli +2
In this paper, we study the compression of a target two-layer neural network with N nodes into a compressed network with M<N nodes. More precisely, we consider the setting in which…
Polar Coded Computing: The Role of the Scaling Exponent
Dorsa Fathollahi, Marco Mondelli
We consider the problem of coded distributed computing using polar codes. The average execution time of a coded computing system is related to the error probability for transmissio…