1 paper
Megan Frisella, Shubham Tiwari, Andy Ruan +5
Large-scale model training increasingly relies on composing multiple parallelism strategies, such as data, pipeline, and expert parallelism, together with memory-saving optimizatio…