4 papers
The Implicit Bias of Depth: From Neural Collapse to Softmax Codes
Connall Garrod, Jonathan P. Keating, Christos Thrampoulidis
Neural collapse (NC) describes the structured geometry that emerges in the features and weights of trained classifiers. Recent theory suggests NC can be suboptimal in deep architec…
When Stronger Triggers Backfire: A High-Dimensional Theory of Backdoor Attacks
Donald Flynn, Hadas Yaron Goldhirsh, Jonathan P. Keating +1
Backdoor poisoning attacks behave counter-intuitively in high dimensions: stronger training triggers can help the defender. We study regularised generalised linear models on Gaussi…
Diagonalizing the Softmax: Hadamard Initialization for Tractable Cross-Entropy Dynamics
Connall Garrod, Jonathan P. Keating, Christos Thrampoulidis
Cross-entropy (CE) training loss dominates deep learning practice, yet existing theory often relies on simplifications, either replacing it with squared loss or restricting to conv…
The Persistence of Neural Collapse Despite Low-Rank Bias
Connall Garrod, Jonathan P. Keating
Neural collapse (NC) and its multi-layer variant, deep neural collapse (DNC), describe a structured geometry that occurs in the features and weights of trained deep networks. Recen…