16.2k citations · 18.3k across the 11 of their papers we have counts for
3 papers · 1 filter
Automap: Towards Ergonomic Automated Parallelism for ML Models
Michael Schaarschmidt, Dominik Grewe, Dimitrios Vytiniotis +8
The rapid rise in demand for training large neural network architectures has brought into focus the need for partitioning strategies, for example by using data, model, or pipeline…
Memory-efficient array redistribution through portable collective communication
Norman A. Rink, Adam Paszke, Dimitrios Vytiniotis +1
Modern large-scale deep learning workloads highlight the need for parallel execution across many devices in order to fit model data into hardware accelerator memories. In these set…
Tensors Fitting Perfectly
Adam Paszke, Brennan Saeta
Multidimensional arrays (NDArrays) are a central abstraction in modern scientific computing environments. Unfortunately, they can make reasoning about programs harder as the number…