activity
20172025
most citedScalable and Efficient MoE Training for Multitask Multilingual Models

33 citations · 63 across the 7 of their papers we have counts for

collaborators
Showing cs.LGShow all

1 paper · 1 filter

cs.LG2024

u-P: The Unit-Scaled Maximal Update Parametrization

Charlie Blake, Constantin Eichenberg, Josef Dean +7

The Maximal Update Parametrization (P) aims to make the optimal hyperparameters (HPs) of a model independent of its size, allowing them to be swept using a cheap proxy model rat…