Spectral Condition for $\mu$P under Width-Depth Scaling

Chenyu Zheng, Rongzhen Wang, Xinyu Zhang +1

Generative foundation models are increasingly scaled in both width and depth, posing significant challenges for stable feature learning and reliable hyperparameter (HP) transfer ac…