4 citations · 4 across the 3 of their papers we have counts for
3 papers · 1 filter
INDIGO: Page Migration for Hardware Memory Disaggregation Across a Network
Archit Patke, Christian Pinto, Saurabh Jha +3
Hardware memory disaggregation (HMD) is an emerging technology that enables access to remote memory, thereby creating expansive memory pools and reducing memory underutilization in…
Queue management for slo-oriented large language model serving
Archit Patke, Dhemath Reddy, Saurabh Jha +5
Large language model (LLM) serving is becoming an increasingly critical workload for cloud providers. Existing LLM serving systems focus on interactive requests, such as chatbots a…
Efficient Interactive LLM Serving with Proxy Model-based Sequence Length Prediction
Haoran Qiu, Weichao Mao, Archit Patke +7
Large language models (LLMs) have been driving a new wave of interactive AI applications across numerous domains. However, efficiently serving LLM inference requests is challenging…