15 citations · 18 across the 4 of their papers we have counts for
4 papers
Small Singular Values Matter: A Random Matrix Analysis of Transformer Models
Max Staats, Matthias Thamm, Bernd Rosenow
This work analyzes singular-value spectra of weight matrices in pretrained transformer models to understand how information is stored at both ends of the spectrum. Using Random Mat…
Enhancing Noise-Robust Losses for Large-Scale Noisy Data Learning
Max Staats, Matthias Thamm, Bernd Rosenow
Large annotated datasets inevitably contain noisy labels, which poses a major challenge for training deep neural networks as they easily memorize the labels. Noise-robust loss func…
Boundary between noise and information applied to filtering neural network weight matrices
Max Staats, Matthias Thamm, Bernd Rosenow
Deep neural networks have been successfully applied to a broad range of problems where overparametrization yields weight matrices which are partially random. A comparison of weight…
Random matrix analysis of deep neural network weight matrices
Matthias Thamm, Max Staats, Bernd Rosenow
Neural networks have been used successfully in a variety of fields, which has led to a great deal of interest in developing a theoretical understanding of how they store the inform…