Average gradient outer product as a mechanism for deep neural collapse

Daniel Beaglehole, Peter Súkeník, Marco Mondelli +1

Deep Neural Collapse (DNC) refers to the surprisingly rigid structure of the data representations in the final layers of Deep Neural Networks (DNNs). Though the phenomenon has been…