3 papers
cs.DC2026
Scaling LLM Inference Beyond Amdahl`s Limits via Eliminating Non-Scalable Overheads
Alan Zhao, Cyril Y. He, Wei Xu
Deployers of online LLM services usually seek to maximize cluster-wide performance given a fixed number of GPUs. Tensor parallelism (TP) is necessary to fit modern models but scale…
cs.DC2026
SiDP: Memory-Efficient Data Parallelism for Offline LLM Inference
Alan Zhao, Cyril Y. He
The rapid adoption of large language models (LLMs) has shifted a substantial portion of inference workloads into throughput-oriented offline regimes, where fully utilizing GPU comp…
cs.LG2026
FluxMoE: Decoupling Expert Residency for High-Performance MoE Serving
Qingxiu Liu, Cyril Y. He, Hanser Jiang +3
Mixture-of-Experts (MoE) models have become a dominant paradigm for scaling large language models, but their rapidly growing parameter sizes introduce a fundamental inefficiency du…