29 citations · 31 across the 3 of their papers we have counts for
5 papers
Automatic Discovery of Composite SPMD Partitioning Strategies in PartIR
Sami Alabed, Dominik Grewe, Juliana Franco +6
Large neural network models are commonly trained through a combination of advanced parallelism strategies in a single program, multiple data (SPMD) paradigm. For example, training…
Gradient Forward-Propagation for Large-Scale Temporal Video Modelling
Mateusz Malinowski, Dimitrios Vytiniotis, Grzegorz Swirszcz +2
How can neural networks be trained on large-volume temporal data efficiently? To compute the gradients required to update parameters, backpropagation blocks computations until the…
Getting to the Point. Index Sets and Parallelism-Preserving Autodiff for Pointful Array Programming
Adam Paszke, Daniel Johnson, David Duvenaud +5
We present a novel programming language design that attempts to combine the clarity and safety of high-level functional languages with the efficiency and parallelism of low-level n…
Efficient Differentiable Programming in a Functional Array-Processing Language
Amir Shaikhha, Andrew Fitzgibbon, Dimitrios Vytiniotis +2
We present a system for the automatic differentiation of a higher-order functional array-processing language. The core functional language underlying this system simultaneously sup…
AMPNet: Asynchronous Model-Parallel Training for Dynamic Neural Networks
Alexander L. Gaunt, Matthew A. Johnson, Maik Riechert +4
New types of machine learning hardware in development and entering the market hold the promise of revolutionizing deep learning in a manner as profound as GPUs. However, existing s…