4 citations · 7 across the 6 of their papers we have counts for
Showing 2024Show all
2 papers · 1 filter
cs.DC2024
Queue management for slo-oriented large language model serving
Archit Patke, Dhemath Reddy, Saurabh Jha +5
Large language model (LLM) serving is becoming an increasingly critical workload for cloud providers. Existing LLM serving systems focus on interactive requests, such as chatbots a…
cs.DC2024★ 4 cited
Efficient Interactive LLM Serving with Proxy Model-based Sequence Length Prediction
Haoran Qiu, Weichao Mao, Archit Patke +7
Large language models (LLMs) have been driving a new wave of interactive AI applications across numerous domains. However, efficiently serving LLM inference requests is challenging…