25 citations · 30 across the 10 of their papers we have counts for
3 papers · 1 filter
Characterizing LLM Kernel Access and Memory Interaction in Multi-Partition NUMA GPUs
Donghyeon Joo, Sooraj Puthoor, Nuwan Jayasena +1
Large language model (LLM) workloads motivate multi-partition GPUs as a path to scaling compute and memory capacity, but their non-uniform memory access characteristics and inter-p…
The Inequality: Quantifying the Double Penalty of Mixture-of-Experts at Inference
Vignesh Adhinarayanan, Nuwan Jayasena
Mixture-of-Experts (MoE) models deliver high quality at low training FLOPs, but this efficiency often vanishes at inference. We identify a double penalty that structurally disadvan…
RAPID-Serve: Resource-efficient and Accelerated P/D Intra-GPU Disaggregation
Amna Masood, Pratishtha Gaur, Nuwan Jayasena
Two widely adopted techniques for LLM inference serving systems today are hybrid batching and disaggregated serving. A hybrid batch combines prefill and decode tokens of different…