Neural Collapse versus Low-rank Bias: Is Deep Neural Collapse Really Optimal?

Peter Súkeník, Marco Mondelli, Christoph Lampert

Deep neural networks (DNNs) exhibit a surprising structure in their final layer known as neural collapse (NC), and a growing body of works has currently investigated the propagatio…