5 papers
Cascading Through the Hierarchy: Regularizer-Induced Feature Detection as Phase Transitions in Deep Linear Neural Networks
Björn Ladewig, Ibrahim Talha Ersoy, Karoline Wiesner
A scientific theory of deep learning, comprising learning dynamics and statistical properties of learned models, is rapidly gaining attention. One of the corner stones of this deve…
Noise-Driven Escape from Metastable Phases explains Grokking in Deep Neural Networks
Ibrahim Talha Ersoy, Karoline Wiesner
Deep neural networks (DNNs) exhibit first order phase transitions under variations of the L2 regularization strength, with each transition marking the onset of a new learnable feat…
Phase transitions reveal hierarchical structure in deep neural networks
Ibrahim Talha Ersoy, Andrés Fernando Cardozo Licha, Karoline Wiesner
Training Deep Neural Networks relies on the model converging on a high-dimensional, non-convex loss landscape toward a good minimum. Yet, much of the phenomenology of training rema…
Phase Transitions between Accuracy Regimes in L2 regularized Deep Neural Networks
Ibrahim Talha Ersoy, Karoline Wiesner
Increasing the L2 regularization of Deep Neural Networks (DNNs) causes a first-order phase transition into the under-parametrized phase -- the so-called onset-of learning. We expla…
Holography as Homotopy
Christoph Chiaffrino, Talha Ersoy, Olaf Hohm
We give an interpretation of holography in the form of the AdS/CFT correspondence in terms of homotopy algebras. A field theory such as a bulk gravity theory can be viewed as a hom…