2 papers
cs.LG2025
Small Singular Values Matter: A Random Matrix Analysis of Transformer Models
Max Staats, Matthias Thamm, Bernd Rosenow
This work analyzes singular-value spectra of weight matrices in pretrained transformer models to understand how information is stored at both ends of the spectrum. Using Random Mat…
cs.LG2025
Enhancing Noise-Robust Losses for Large-Scale Noisy Data Learning
Max Staats, Matthias Thamm, Bernd Rosenow
Large annotated datasets inevitably contain noisy labels, which poses a major challenge for training deep neural networks as they easily memorize the labels. Noise-robust loss func…