The Implicit Bias of Depth: From Neural Collapse to Softmax Codes

Connall Garrod, Jonathan P. Keating, Christos Thrampoulidis

Neural collapse (NC) describes the structured geometry that emerges in the features and weights of trained classifiers. Recent theory suggests NC can be suboptimal in deep architec…