343 citations · 369 across the 6 of their papers we have counts for
4 papers · 1 filter
TOAST: Fast and scalable auto-partitioning based on principled static analysis
Sami Alabed, Dominik Grewe, Norman Alexander Rink +5
Partitioning large machine learning models across distributed accelerator systems is a complex process, requiring a series of interdependent decisions that are further complicated…
PartIR: Composing SPMD Partitioning Strategies for Machine Learning
Sami Alabed, Daniel Belov, Bart Chrzaszcz +14
Training of modern large neural networks (NN) requires a combination of parallelization strategies encompassing data, model, or optimizer sharding. When strategies increase in comp…
TF-Replicator: Distributed Machine Learning for Researchers
Peter Buchlovsky, David Budden, Dominik Grewe +9
We describe TF-Replicator, a framework for distributed machine learning designed for DeepMind researchers and implemented as an abstraction over TensorFlow. TF-Replicator simplifie…
Parallel WaveNet: Fast High-Fidelity Speech Synthesis
Aaron van den Oord, Yazhe Li, Igor Babuschkin +19
The recently-developed WaveNet architecture is the current state of the art in realistic speech synthesis, consistently rated as more natural sounding for many different languages…