6 papers
Geometric and Information Compression of Representations in Deep Learning
Linara Adilova, Henning Petzka, Asja Fischer +1
Deep neural networks transform input data into latent representations that support a wide range of downstream tasks. These representations can be characterized along information-th…
Flatness is Necessary, Neural Collapse is Not: Rethinking Generalization via Grokking
Ting Han, Linara Adilova, Henning Petzka +2
Neural collapse, i.e., the emergence of highly symmetric, class-wise clustered representations, is frequently observed in deep networks and is often assumed to reflect or enable ge…
When Flatness Does (Not) Guarantee Adversarial Robustness
Nils Philipp Walter, Linara Adilova, Jilles Vreeken +1
Despite their empirical success, neural networks remain vulnerable to small, adversarial perturbations. A longstanding hypothesis suggests that flat minima, regions of low curvatur…
Fisher information flow in artificial neural networks
Maximilian Weimar, Lukas M. Rachbauer, Ilya Starshynov +4
The estimation of continuous parameters from measured data plays a central role in many fields of physics. A key tool in understanding and improving such estimation processes is th…
The Uncanny Valley: Exploring Adversarial Robustness from a Flatness Perspective
Nils Philipp Walter, Linara Adilova, Jilles Vreeken +1
Flatness of the loss surface not only correlates positively with generalization, but is also related to adversarial robustness since perturbations of inputs relate non-linearly to…
Landscaping Linear Mode Connectivity
Sidak Pal Singh, Linara Adilova, Michael Kamp +3
The presence of linear paths in parameter space between two different network solutions in certain cases, i.e., linear mode connectivity (LMC), has garnered interest from both theo…