4 papers
Randomly initialized autoencoders: fixed points and edge-of-chaos
Leonid Berlyand, Roman Sarapin, Yitzchak Shmalo +2
In this paper we study autoencoders, a special class of deep neural nets (DNNs) whose performance can be characterized via their fixed points. This perspective naturally raises que…
Pruning Deep Neural Networks via the Marchenko--Pastur Distribution
Leonid Berlyand, Theo Bourdais, Houman Owhadi +1
We study a Marchenko--Pastur (MP) random-matrix approach to pruning deep neural networks with very small post-pruning fine-tuning budgets. The main practical contribution is accura…
Random weights of DNNs and emergence of fixed points
L. Berlyand, O. Krupchytskyi, V. Slavin
This paper is concerned with a special class of deep neural networks (DNNs) where the input and the output vectors have the same dimension. Such DNNs are widely used in application…
Pruning Deep Neural Networks via a Combination of the Marchenko-Pastur Distribution and Regularization
Leonid Berlyand, Theo Bourdais, Houman Owhadi +1
Deep neural networks (DNNs) have brought significant advancements in various applications in recent years, such as image recognition, speech recognition, and natural language proce…