3 papers
cs.DC2026
CTA-Pipelining: A Latency-Oriented Spatial Scaling Method for Multi-GPU Systems
Tingkai Liu, Muralidhar Andoorveedu, Sanjoy Das +2
The evolution of compute infrastructure has transformed multi-GPU systems into tightly integrated shared-memory structures. However, current software still mostly treats these cohe…
cs.DC2025
Mist: Efficient Distributed Training of Large Language Models via Memory-Parallelism Co-Optimization
Zhanda Zhu, Christina Giannoula, Muralidhar Andoorveedu +4
Various parallelism, such as data, tensor, and pipeline parallelism, along with memory optimizations like activation checkpointing, redundancy elimination, and offloading, have bee…
cs.DC2025
Seesaw: High-throughput LLM Inference via Model Re-sharding
Qidong Su, Wei Zhao, Xin Li +6
To improve the efficiency of distributed large language model (LLM) inference, various parallelization strategies, such as tensor and pipeline parallelism, have been proposed. Howe…