24 citations · 27 across the 17 of their papers we have counts for
8 papers · 1 filter
Neural means and kernel corrections for operator learning
Yitzchak Shmalo
We combine neural network means with exact Matérn kernel regressions of their residuals and of their learned features, and evaluate the pairing on two public emulation problems wit…
Randomly initialized autoencoders: fixed points and edge-of-chaos
Leonid Berlyand, Roman Sarapin, Yitzchak Shmalo +2
In this paper we study autoencoders, a special class of deep neural nets (DNNs) whose performance can be characterized via their fixed points. This perspective naturally raises que…
A law of robustness for two-layer neural networks with arbitrary weights
Yitzchak Shmalo
Bubeck, Li and Nagaraj conjectured that, for generic data, any two-layer neural network with neurons that fits noisy labels must have Lipschitz constant at least of order $…
Pruning Deep Neural Networks via the Marchenko--Pastur Distribution
Leonid Berlyand, Theo Bourdais, Houman Owhadi +1
We study a Marchenko--Pastur (MP) random-matrix approach to pruning deep neural networks with very small post-pruning fine-tuning budgets. The main practical contribution is accura…
Pruning Deep Neural Networks via a Combination of the Marchenko-Pastur Distribution and Regularization
Leonid Berlyand, Theo Bourdais, Houman Owhadi +1
Deep neural networks (DNNs) have brought significant advancements in various applications in recent years, such as image recognition, speech recognition, and natural language proce…
Enhancing Accuracy in Deep Learning Using Random Matrix Theory
Leonid Berlyand, Etienne Sandier, Yitzchak Shmalo +1
We explore the applications of random matrix theory (RMT) in the training of deep neural networks (DNNs), focusing on layer pruning that is reducing the number of DNN parameters (w…
Deep Learning Weight Pruning with RMT-SVD: Increasing Accuracy and Reducing Overfitting
Yitzchak Shmalo, Jonathan Jenkins, Oleksii Krupchytskyi
In this work, we present some applications of random matrix theory for the training of deep neural networks. Recently, random matrix theory (RMT) has been applied to the overfittin…
Stability of Accuracy for the Training of DNNs Via the Uniform Doubling Condition
Yitzchak Shmalo
We study the stability of accuracy during the training of deep neural networks (DNNs). In this context, the training of a DNN is performed via the minimization of a cross-entropy l…