25 citations · 30 across the 10 of their papers we have counts for
10 papers
Characterizing LLM Kernel Access and Memory Interaction in Multi-Partition NUMA GPUs
Donghyeon Joo, Sooraj Puthoor, Nuwan Jayasena +1
Large language model (LLM) workloads motivate multi-partition GPUs as a path to scaling compute and memory capacity, but their non-uniform memory access characteristics and inter-p…
The Inequality: Quantifying the Double Penalty of Mixture-of-Experts at Inference
Vignesh Adhinarayanan, Nuwan Jayasena
Mixture-of-Experts (MoE) models deliver high quality at low training FLOPs, but this efficiency often vanishes at inference. We identify a double penalty that structurally disadvan…
RAPID-Serve: Resource-efficient and Accelerated P/D Intra-GPU Disaggregation
Amna Masood, Pratishtha Gaur, Nuwan Jayasena
Two widely adopted techniques for LLM inference serving systems today are hybrid batching and disaggregated serving. A hybrid batch combines prefill and decode tokens of different…
Global Optimizations & Lightweight Dynamic Logic for Concurrency
Suchita Pati, Shaizeen Aga, Nuwan Jayasena +1
Modern accelerators like GPUs are increasingly executing independent operations concurrently to improve the device's compute utilization. However, effectively harnessing it on GPUs…
T3: Transparent Tracking & Triggering for Fine-grained Overlap of Compute & Collectives
Suchita Pati, Shaizeen Aga, Mahzabeen Islam +2
Large Language Models increasingly rely on distributed techniques for their training and inference. These techniques require communication across devices which can reduce scaling e…
Inclusive-PIM: Hardware-Software Co-design for Broad Acceleration on Commercial PIM Architectures
Johnathan Alsop, Shaizeen Aga, Mohamed Ibrahim +3
Continual demand for memory bandwidth has made it worthwhile for memory vendors to reassess processing in memory (PIM), which enables higher bandwidth by placing compute units in/n…
Computation vs. Communication Scaling for Future Transformers on Future Hardware
Suchita Pati, Shaizeen Aga, Mahzabeen Islam +2
Scaling neural network models has delivered dramatic quality gains across ML problems. However, this scaling has increased the reliance on efficient distributed training techniques…
Demystifying BERT: Implications for Accelerator Design
Suchita Pati, Shaizeen Aga, Nuwan Jayasena +1
Transfer learning in natural language processing (NLP), as realized using models like BERT (Bi-directional Encoder Representation from Transformer), has significantly improved lang…
SeqPoint: Identifying Representative Iterations of Sequence-based Neural Networks
Suchita Pati, Shaizeen Aga, Matthew D. Sinclair +1
The ubiquity of deep neural networks (DNNs) continues to rise, making them a crucial application class for hardware optimizations. However, detailed profiling and characterization…
CODA: Enabling Co-location of Computation and Data for Near-Data Processing
Hyojong Kim, Ramyad Hadidi, Lifeng Nai +5
Recent studies have demonstrated that near-data processing (NDP) is an effective technique for improving performance and energy efficiency of data-intensive workloads. However, lev…