21 citations · 54 across the 11 of their papers we have counts for
11 papers
PartIR: Composing SPMD Partitioning Strategies for Machine Learning
Sami Alabed, Daniel Belov, Bart Chrzaszcz +14
Training of modern large neural networks (NN) requires a combination of parallelization strategies encompassing data, model, or optimizer sharding. When strategies increase in comp…
Automatic Discovery of Composite SPMD Partitioning Strategies in PartIR
Sami Alabed, Dominik Grewe, Juliana Franco +6
Large neural network models are commonly trained through a combination of advanced parallelism strategies in a single program, multiple data (SPMD) paradigm. For example, training…
Automap: Towards Ergonomic Automated Parallelism for ML Models
Michael Schaarschmidt, Dominik Grewe, Dimitrios Vytiniotis +8
The rapid rise in demand for training large neural network architectures has brought into focus the need for partitioning strategies, for example by using data, model, or pipeline…
Memory-efficient array redistribution through portable collective communication
Norman A. Rink, Adam Paszke, Dimitrios Vytiniotis +1
Modern large-scale deep learning workloads highlight the need for parallel execution across many devices in order to fit model data into hardware accelerator memories. In these set…
Category-Theoretic Foundations of "STCLang: State Thread Composition as a Foundation for Monadic Dataflow Parallelism"
Sebastian Ertel, Justus Adam, Norman A. Rink +2
This manuscript gives a category-theoretic foundation to the composition of State Threads as a Foundation for Monadic Dataflow Parallelism. It serves as a supplementary formalizati…
Modeling of languages for tensor manipulation
Norman A. Rink
Numerical applications and, more recently, machine learning applications rely on high-dimensional data that is typically organized into multi-dimensional tensors. Many existing fra…
A ladder of topologically non-trivial non-BPS states
Daniele Dorigoni, Norman A. Rink
We consider a simple quiver gauge theory with gauge group U(r1) x U(r2) and a Higgs field in the bi-fundamental representation. The back-ground for this theory is a compact Kähler…
Vortices and the Abel-Jacobi map
Norman A. Rink
The abelian Higgs model on a compact Riemann surface Σsupports vortex solutions for any positive vortex number d \in \ZZ. Moreover, the vortex moduli space for fixed d has long bee…
Non-abelian vortices on CP^1 and Grassmannians
Norman A. Rink
Many properties of the moduli space of abelian vortices on a compact Riemann surface are known. For non-abelian vortices the moduli space is less well understood. Here we consider…
Geometry and Energy of Non-abelian Vortices
Nicholas S. Manton, Norman A. Rink
We study pure Yang--Mills theory on , where is a compact Riemann surface, and invariance is assumed under rotations of . It is well known that the self-dualit…
Vortices on Hyperbolic Surfaces
Nicholas S. Manton, Norman A. Rink
It is shown that abelian Higgs vortices on a hyperbolic surface can be constructed geometrically from holomorphic maps , where is also a hyperbolic surface. The…