45 citations · 105 across the 21 of their papers we have counts for
6 papers · 1 filter
COPUS: Co-adaptive Parallelism and Batch Size Selection in Large Language Model Training
Akhmed Sakip, Erland Hilman Fuadi, Omar Sayedelahl +6
Training large language models requires jointly configuring two interdependent aspects of the system: the global batch size, which governs statistical efficiency, and the 3D parall…
LAPS: A Length-Aware-Prefill LLM Serving System
Jianshu She, Zonghang Li, Hongchao Du +7
LAPS identifies and disaggregates requests with different prompt lengths in LLM serving to reduce TTFT latency. While recent systems have decoupled the prefill and decode stages to…
Learning In Chaos: Efficient Autoscaling and Self-Healing for Multi-Party Distributed Training
Wenjiao Feng, Rongxing Xiao, Zonghang Li +6
Node and link churn in multi-party, cross-region clusters over wide-area networks (WANs) often disrupts distributed training. However, checkpoint-based recovery and cloud-centric a…
Prima.cpp: Fast 30-70B LLM Inference on Heterogeneous and Low-Resource Home Clusters
Zonghang Li, Tao Li, Wenjiao Feng +8
On-device inference offers privacy, offline use, and instant response, but consumer hardware restricts large language models (LLMs) to low throughput and capability. To overcome th…
TPI-LLM: Serving 70B-scale LLMs Efficiently on Low-resource Edge Devices
Zonghang Li, Wenjiao Feng, Mohsen Guizani +1
Large model inference is shifting from cloud to edge due to concerns about the privacy of user interaction data. However, edge devices often struggle with limited computing power,…
Accelerating Geo-distributed Machine Learning with Network-Aware Adaptive Tree and Auxiliary Route
Zonghang Li, Wenjiao Feng, Weibo Cai +5
Distributed machine learning is becoming increasingly popular for geo-distributed data analytics, facilitating the collaborative analysis of data scattered across data centers in d…