Showing 2024Show all
2 papers · 1 filter
cs.CE2024
Task Scheduling for Efficient Inference of Large Language Models on Single Moderate GPU Systems
Wenxiang Lin, Xinglin Pan, Shaohuai Shi +2
Large language models~(LLMs) are known for their high demand on computing resources and memory due to their substantial model size, which leads to inefficient inference on moderate…
cs.DC2024
Scheduling Deep Learning Jobs in Multi-Tenant GPU Clusters via Wise Resource Sharing
Yizhou Luo, Qiang Wang, Shaohuai Shi +4
Deep learning (DL) has demonstrated significant success across diverse fields, leading to the construction of dedicated GPU accelerators within GPU clusters for high-quality traini…