Deep Neural Collapse Is Provably Optimal for the Deep Unconstrained Features Model

Peter Súkeník, Marco Mondelli, Christoph Lampert

Neural collapse (NC) refers to the surprising structure of the last layer of deep neural networks in the terminal phase of gradient descent training. Recently, an increasing amount…