A Boundary-Layer Mechanism for One-Third Scaling in Online Softmax Classification

Marcel Kühn, Yoon Thelge, Bernd Rosenow

Hard-label classification is usually trained with smooth surrogate losses such as softmax cross-entropy. We isolate a mechanism by which this mismatch between smooth surrogate and…