Scaling Up Models and Data with $\text{t5x}$ and $\text{seqio}$

Adam Roberts, Hyung Won Chung, Anselm Levskaya +40

Recent neural network-based language models have benefited greatly from scaling up the size of training datasets and the number of parameters in the models themselves. Scaling can…