2 citations · 2 across the 2 of their papers we have counts for
4 papers
Unifying Low Dimensional Spectra in Deep Learning
Connall Garrod, Jonathan P. Keating
Low dimensional structures appear ubiquitously in the eigenspectra of deep learning matrices in classification networks trained in the overparameterized regime. While theoretical a…
The Implicit Bias of Depth: From Neural Collapse to Softmax Codes
Connall Garrod, Jonathan P. Keating, Christos Thrampoulidis
Neural collapse (NC) describes the structured geometry that emerges in the features and weights of trained classifiers. Recent theory suggests NC can be suboptimal in deep architec…
Diagonalizing the Softmax: Hadamard Initialization for Tractable Cross-Entropy Dynamics
Connall Garrod, Jonathan P. Keating, Christos Thrampoulidis
Cross-entropy (CE) training loss dominates deep learning practice, yet existing theory often relies on simplifications, either replacing it with squared loss or restricting to conv…
The Persistence of Neural Collapse Despite Low-Rank Bias
Connall Garrod, Jonathan P. Keating
Neural collapse (NC) and its multi-layer variant, deep neural collapse (DNC), describe a structured geometry that occurs in the features and weights of trained deep networks. Recen…