DynaPipe: Optimizing Multi-task Training through Dynamic Pipelines

Chenyu Jiang, Zhen Jia, Shuai Zheng +2

Multi-task model training has been adopted to enable a single deep neural network model (often a large language model) to handle multiple tasks (e.g., question answering and text s…