Showing cs.DCShow all
2 papers · 1 filter
cs.DC2026
Koi: Evidence-Guided, Self-Calibrating Planning for Multi-Cloud, Heterogeneous LLM Inference Fleets
Mankeerat Sidhu, Gangmuk Lim, Hetarth Chopra +10
Deploying a large language model (LLM) requires choosing the right accelerator, parallelization strategy, engine configuration, and replica count amongst ~140 other variables acros…
cs.DC2026
Fast Recovery for LLM Serving via Decoupled Device Memory Lifetime in Dynamo
Schwinn Saereesitthipitak, Mohammed Abdulwahhab, Hannah Zhang +6
Large language model (LLM) inference replicas run across tightly coupled GPUs and serve traffic continuously for weeks. Hardware and software failures are therefore inevitable, and…