5 papers
Scaling LLM Inference Beyond Amdahl`s Limits via Eliminating Non-Scalable Overheads
Alan Zhao, Cyril Y. He, Wei Xu
Deployers of online LLM services usually seek to maximize cluster-wide performance given a fixed number of GPUs. Tensor parallelism (TP) is necessary to fit modern models but scale…
A Readiness-Driven Runtime for Pipeline-Parallel Training under Runtime Variability
Ruitao Liu, Xinyang Tian, Shuo Chen +4
Pipeline parallelism is a key technique for scaling large-model training, but modern workloads exhibit runtime variability in computation and communication. Existing pipeline syste…
FFTrainer: Fast Failover in Large-Language Model Training with Almost-Free State Management
Bohan Zhao, Yuanhong Wang, Chenglin Liu +6
Recent developments in large language models (LLMs) have introduced new requirements for efficient and robust training. As LLM clusters scale, node failures, lengthy recoveries, an…
Kronos: A Foundation Model for the Language of Financial Markets
Yu Shi, Zongliang Fu, Shuo Chen +4
The success of large-scale pre-training paradigm, exemplified by Large Language Models (LLMs), has inspired the development of Time Series Foundation Models (TSFMs). However, their…
MegatronApp: Efficient and Comprehensive Management on Distributed LLM Training
Bohan Zhao, Guang Yang, Shuo Chen +4
The rapid escalation in the parameter count of large language models (LLMs) has transformed model training from a single-node endeavor into a highly intricate, cross-node activity.…