Every Kernel Is a Join: Automatic Multi-GPU Parallelism for AI Computations in Einsummable

Zhimin Ding, Chen-Kuan Liao, Chima Adiole +7

Distributing an AI computation across the GPUs of a multi-GPU server is one of the central problems in systems-for-AI. We present Einsummable, a prototype system that accepts a PyT…