A Boundary-Layer Mechanism for One-Third Scaling in Online Softmax Classification

Marcel Kühn, Yoon Thelge, Bernd Rosenow

Hard-label classification is usually trained with smooth surrogate losses, most prominently softmax cross-entropy. We isolate an asymptotic mechanism by which this mismatch between…