76 citations · 107 across the 6 of their papers we have counts for
3 papers · 1 filter
Koi: Evidence-Guided, Self-Calibrating Planning for Multi-Cloud, Heterogeneous LLM Inference Fleets
Mankeerat Sidhu, Gangmuk Lim, Hetarth Chopra +10
Deploying a large language model (LLM) requires choosing the right accelerator, parallelization strategy, engine configuration, and replica count amongst ~140 other variables acros…
SafePar: Monitoring Asynchrony in Microservices
Karuna Grewal, P. Brighten Godfrey, Justin Hsu +1
Modern cloud applications are built from loosely-coupled microservices that coordinate through well-defined APIs to service user requests. A single API request often triggers multi…
SafeNom: Data-Aware Microservice Policies
Karuna Grewal, P. Brighten Godfrey, Justin Hsu
Many cloud-based applications are organized as loosely coupled microservices, where invoking a service's API triggers a cascade of APIs across many services and leads to inter-serv…