Showing 2024Show all
2 papers · 1 filter
cs.LG2024
The Persistence of Neural Collapse Despite Low-Rank Bias
Connall Garrod, Jonathan P. Keating
Neural collapse (NC) and its multi-layer variant, deep neural collapse (DNC), describe a structured geometry that occurs in the features and weights of trained deep networks. Recen…
cs.LG2024
Unifying Low Dimensional Spectra in Deep Learning
Connall Garrod, Jonathan P. Keating
Low dimensional structures appear ubiquitously in the eigenspectra of deep learning matrices in classification networks trained in the overparameterized regime. While theoretical a…