Towards Understanding Neural Collapse: The Effects of Batch Normalization and Weight Decay

Leyan Pan, Xinyuan Cao

Neural Collapse (NC) is a geometric structure recently observed at the terminal phase of training deep neural networks, which states that last-layer feature vectors for the same cl…