Layer-Parallel Training for Transformers

Shuai Jiang, Marc Salvadó-Benasco, Eric C. Cyr +3

We present a new training methodology for transformers using a multilevel, layer-parallel approach. Through a neural ODE formulation of transformers, our application of a multileve…