20 citations · 21 across the 2 of their papers we have counts for
2 papers
cs.LG2024★ 20 cited
POD-Attention: Unlocking Full Prefill-Decode Overlap for Faster LLM Inference
Aditya K Kamath, Ramya Prabhu, Jayashree Mohan +3
Each request in LLM inference goes through two phases: compute-bound prefill and memory-bandwidth-bound decode. To improve GPU utilization, recent systems use hybrid batching that…
cs.OS2023★ 1 cited
MaxMem: Colocation and Performance for Big Data Applications on Tiered Main Memory Servers
Amanda Raybuck, Wei Zhang, Kayvan Mansoorshahi +3
We present MaxMem, a tiered main memory management system that aims to maximize Big Data application colocation and performance. MaxMem uses an application-agnostic and lightweight…