Showing cs.LGShow all
2 papers · 1 filter
cs.LG2026
Hyperparameter Transfer Laws for Non-Recurrent Multi-Path Neural Networks
Shenxi Wu, Haosong Zhang, Xingjian Ma +4
Deeper modern architectures are costly to train, making hyperparameter transfer preferable to expensive repeated tuning. Maximal Update Parametrization (P) helps explain why man…
cs.LG2025
Arithmetic-Mean P for Modern Architectures: A Unified Learning-Rate Scale for CNNs and ResNets
Haosong Zhang, Shenxi Wu, Yichi Zhang +2
Choosing an appropriate learning rate remains a key challenge in scaling depth of modern deep networks. The classical maximal update parameterization (P) enforces a fixed per-la…