106 citations · 167 across the 26 of their papers we have counts for
6 papers · 1 filter
HashAttention: Semantic Sparsity for Faster Inference
Aditya Desai, Shuo Yang, Alejandro Cuadron +3
Leveraging long contexts is crucial for advanced AI systems, but attention computation poses a scalability challenge. While scaled dot-product attention (SDPA) exhibits token spars…
Imaginary Machines: A Serverless Model for Cloud Applications
Michael Wawrzoniak, Rodrigo Bruno, Ana Klimovic +1
Serverless Function-as-a-Service (FaaS) platforms provide applications with resources that are highly elastic, quick to instantiate, accounted at fine granularity, and without the…
Boxer: FaaSt Ephemeral Elasticity for Off-the-Shelf Cloud Applications
Michael Wawrzoniak, Rodrigo Bruno, Ana Klimovic +1
Elasticity is a key property of cloud computing. However, elasticity is offered today at the granularity of virtual machines, which take tens of seconds to start. This is insuffici…
Dirigent: Lightweight Serverless Orchestration
Lazar Cvetković, François Costa, Mihajlo Djokic +2
While Function as a Service (FaaS) platforms can initialize function sandboxes on worker nodes in 10-100s of milliseconds, the latency to schedule functions in real FaaS clusters c…
DéjàVu: KV-cache Streaming for Fast, Fault-tolerant Generative LLM Serving
Foteini Strati, Sara Mcallister, Amar Phanishayee +2
Distributed LLM serving is costly and often underutilizes hardware accelerators due to three key challenges: bubbles in pipeline-parallel deployments caused by the bimodal latency…
On Distributed Larger-Than-Memory Subset Selection With Pairwise Submodular Functions
Maximilian Böther, Abraham Sebastian, Pranjal Awasthi +2
Modern datasets span billions of samples, making training on all available data infeasible. Selecting a high quality subset helps in reducing training costs and enhancing model qua…