239 citations · 279 across the 6 of their papers we have counts for
6 papers
Towards Greener LLMs: Bringing Energy-Efficiency to the Forefront of LLM Inference
Jovan Stojkovic, Esha Choukse, Chaojie Zhang +2
With the ubiquitous use of modern large language models (LLMs) across industries, the inference serving for these models is ever expanding. Given the high compute and memory requir…
Splitwise: Efficient generative LLM inference using phase splitting
Pratyush Patel, Esha Choukse, Chaojie Zhang +4
Recent innovations in generative large language models (LLMs) have made their applications and use-cases ubiquitous. This has led to large-scale deployments of these models, using…
POLCA: Power Oversubscription in LLM Cloud Providers
Pratyush Patel, Esha Choukse, Chaojie Zhang +4
Recent innovation in large language models (LLMs), and their myriad use-cases have rapidly driven up the compute capacity demand for datacenter GPUs. Several cloud providers and ot…
Faa$T: A Transparent Auto-Scaling Cache for Serverless Applications
Francisco Romero, Gohar Irfan Chaudhry, Íñigo Goiri +6
Function-as-a-Service (FaaS) has become an increasingly popular way for users to deploy their applications without the burden of managing the underlying infrastructure. However, ex…
Serverless in the Wild: Characterizing and Optimizing the Serverless Workload at a Large Cloud Provider
Mohammad Shahrad, Rodrigo Fonseca, Íñigo Goiri +7
Function as a Service (FaaS) has been gaining popularity as a way to deploy computations to serverless backends in the cloud. This paradigm shifts the complexity of allocating and…
C2MS: Dynamic Monitoring and Management of Cloud Infrastructures
Gary A. McGilvary, Josep Rius, Íñigo Goiri +3
Server clustering is a common design principle employed by many organisations who require high availability, scalability and easier management of their infrastructure. Servers are…