4 papers · 1 filter
Randomly initialized autoencoders: fixed points and edge-of-chaos
Leonid Berlyand, Roman Sarapin, Yitzchak Shmalo +2
In this paper we study autoencoders, a special class of deep neural nets (DNNs) whose performance can be characterized via their fixed points. This perspective naturally raises que…
Pruning Deep Neural Networks via the Marchenko--Pastur Distribution
Leonid Berlyand, Theo Bourdais, Houman Owhadi +1
We study a Marchenko--Pastur (MP) random-matrix approach to pruning deep neural networks with very small post-pruning fine-tuning budgets. The main practical contribution is accura…
Pruning Deep Neural Networks via a Combination of the Marchenko-Pastur Distribution and Regularization
Leonid Berlyand, Theo Bourdais, Houman Owhadi +1
Deep neural networks (DNNs) have brought significant advancements in various applications in recent years, such as image recognition, speech recognition, and natural language proce…
Enhancing Accuracy in Deep Learning Using Random Matrix Theory
Leonid Berlyand, Etienne Sandier, Yitzchak Shmalo +1
We explore the applications of random matrix theory (RMT) in the training of deep neural networks (DNNs), focusing on layer pruning that is reducing the number of DNN parameters (w…