2 papers
cs.DC2025
Characterizing Communication Patterns in Distributed Large Language Model Inference
Lang Xu, Kaushik Kandadi Suresh, Quentin Anthony +2
Large Language Models (LLMs) built on transformer architectures have transformed natural language processing, achieving remarkable performance across diverse applications. While di…
cs.DC2025
Scaling Large Language Model Training on Frontier with Low-Bandwidth Partitioning
Lang Xu, Quentin Anthony, Jacob Hatef +4
Scaling up Large Language Model(LLM) training involves fitting a tremendous amount of training parameters across a limited number of workers. However, methods like ZeRO-3 that dras…