Showing cs.DCShow all
3 papers · 1 filter
cs.DC2026
Every Kernel Is a Join: Automatic Multi-GPU Parallelism for AI Computations in Einsummable
Zhimin Ding, Chen-Kuan Liao, Chima Adiole +7
Distributing an AI computation across the GPUs of a multi-GPU server is one of the central problems in systems-for-AI. We present Einsummable, a prototype system that accepts a PyT…
cs.DC2024
EinDecomp: Decomposition of Declaratively-Specified Machine Learning and Numerical Computations for Parallel Execution
Daniel Bourgeois, Zhimin Ding, Dimitrije Jankov +6
We consider the problem of automatically decomposing operations over tensors or arrays so that they can be executed in parallel on multiple devices. We address two, closely-linked…
cs.DC2024
TURNIP: A "Nondeterministic" GPU Runtime with CPU RAM Offload
Zhimin Ding, Jiawen Yao, Brianna Barrow +7
An obvious way to alleviate memory difficulties in GPU-based AI computing is via CPU offload, where data are moved between GPU and CPU RAM, so inexpensive CPU RAM is used to increa…