DistIR: An Intermediate Representation and Simulator for Efficient Neural Network Distribution

Keshav Santhanam, Siddharth Krishna, Ryota Tomioka +2

The rapidly growing size of deep neural network (DNN) models and datasets has given rise to a variety of distribution strategies such as data, tensor-model, pipeline parallelism, a…