most citedTPI-LLM: Serving 70B-scale LLMs Efficiently on Low-resource Edge Devices

1 citations · 2 across the 4 of their papers we have counts for

collaborators
Showing cs.DCShow all

3 papers · 1 filter

cs.DC2025

Learning In Chaos: Efficient Autoscaling and Self-Healing for Multi-Party Distributed Training

Wenjiao Feng, Rongxing Xiao, Zonghang Li +6

Node and link churn in multi-party, cross-region clusters over wide-area networks (WANs) often disrupts distributed training. However, checkpoint-based recovery and cloud-centric a…

cs.DC2025★ 1 cited

Prima.cpp: Fast 30-70B LLM Inference on Heterogeneous and Low-Resource Home Clusters

Zonghang Li, Tao Li, Wenjiao Feng +8

On-device inference offers privacy, offline use, and instant response, but consumer hardware restricts large language models (LLMs) to low throughput and capability. To overcome th…

cs.DC2024★ 1 cited

TPI-LLM: Serving 70B-scale LLMs Efficiently on Low-resource Edge Devices

Zonghang Li, Wenjiao Feng, Mohsen Guizani +1

Large model inference is shifting from cloud to edge due to concerns about the privacy of user interaction data. However, edge devices often struggle with limited computing power,…