2 papers
cs.DC2025
SeaLLM: Service-Aware and Latency-Optimized Resource Sharing for Large Language Model Inference
Yihao Zhao, Jiadun Chen, Peng Sun +3
Large language models (LLMs) with different architectures and sizes have been developed. Serving each LLM with dedicated GPUs leads to resource waste and service inefficiency due t…
cs.DC2024
LoongServe: Efficiently Serving Long-Context Large Language Models with Elastic Sequence Parallelism
Bingyang Wu, Shengyu Liu, Yinmin Zhong +3
The context window of large language models (LLMs) is rapidly increasing, leading to a huge variance in resource usage between different requests as well as between different phase…