1 paper · 2 filters
Megan Frisella, Shubham Tiwari, Andy Ruan +5
Large-scale model training increasingly relies on composing multiple parallelism strategies, such as data, pipeline, and expert parallelism, together with memory-saving optimizatio…