SGD in Multiclass Logistic Regression: Sequential Learning and Scaling Laws

Konstantinos Christopher Tsiolis, Denny Wu, Christos Thrampoulidis +1

We study the training dynamics of multiclass logistic regression on high-dimensional Gaussian mixture models with a large number of classes and establish precise scaling laws gover…