21 citations · 54 across the 11 of their papers we have counts for
Showing cs.DCShow all
2 papers · 1 filter
cs.DC2022★ 1 cited
Automatic Discovery of Composite SPMD Partitioning Strategies in PartIR
Sami Alabed, Dominik Grewe, Juliana Franco +6
Large neural network models are commonly trained through a combination of advanced parallelism strategies in a single program, multiple data (SPMD) paradigm. For example, training…
cs.DC2021★ 1 cited
Memory-efficient array redistribution through portable collective communication
Norman A. Rink, Adam Paszke, Dimitrios Vytiniotis +1
Modern large-scale deep learning workloads highlight the need for parallel execution across many devices in order to fit model data into hardware accelerator memories. In these set…