Parallel Training of GRU Networks with a Multi-Grid Solver for Long Sequences

Gordon Euhyun Moon, Eric C. Cyr

Parallelizing Gated Recurrent Unit (GRU) networks is a challenging task, as the training procedure of GRU is inherently sequential. Prior efforts to parallelize GRU have largely fo…