Showing cs.DCShow all
2 papers · 1 filter
cs.DC2026
Scaling LLM Inference Beyond Amdahl`s Limits via Eliminating Non-Scalable Overheads
Alan Zhao, Cyril Y. He, Wei Xu
Deployers of online LLM services usually seek to maximize cluster-wide performance given a fixed number of GPUs. Tensor parallelism (TP) is necessary to fit modern models but scale…
cs.DC2026
SiDP: Memory-Efficient Data Parallelism for Offline LLM Inference
Alan Zhao, Cyril Y. He
The rapid adoption of large language models (LLMs) has shifted a substantial portion of inference workloads into throughput-oriented offline regimes, where fully utilizing GPU comp…