20 citations · 28 across the 7 of their papers we have counts for
3 papers · 1 filter
POD-Attention: Unlocking Full Prefill-Decode Overlap for Faster LLM Inference
Aditya K Kamath, Ramya Prabhu, Jayashree Mohan +3
Each request in LLM inference goes through two phases: compute-bound prefill and memory-bandwidth-bound decode. To improve GPU utilization, recent systems use hybrid batching that…
LLeMpower: Understanding Disparities in the Control and Access of Large Language Models
Vishwas Sathish, Hannah Lin, Aditya K Kamath +1
Large Language Models (LLMs) are a powerful technology that augment human skill to create new opportunities, akin to the development of steam engines and the internet. However, LLM…
Herding LLaMaS: Using LLMs as an OS Module
Aditya K Kamath, Sujay Yadalam
Computer systems are becoming increasingly heterogeneous with the emergence of new memory technologies and compute devices. GPUs alongside CPUs have become commonplace and CXL is p…