Automatic Operator-level Parallelism Planning for Distributed Deep Learning -- A Mixed-Integer Programming Approach

Ruifeng She, Bowen Pang, Kai Li +2

As the artificial intelligence community advances into the era of large models with billions of parameters, distributed training and inference have become essential. While various…