56 citations · 71 across the 7 of their papers we have counts for
8 papers
Efficient MoE Serving in the Memory-Bound Regime: Balance Activated Experts, Not Tokens
Yanpeng Yu, Haiyue Ma, Krish Agarwal +10
Expert Parallelism (EP) permits Mixture of Experts (MoE) models to scale beyond a single GPU. To address load imbalance across GPUs in EP, existing approaches aim to balance the nu…
Prompt Cache: Modular Attention Reuse for Low-Latency Inference
In Gim, Guojun Chen, Seung-seob Lee +3
We present Prompt Cache, an approach for accelerating inference for large language models (LLM) by reusing attention states across different LLM prompts. Many input prompts have ov…
Karma: Resource Allocation for Dynamic Demands
Midhul Vuppalapati, Giannis Fikioris, Rachit Agarwal +3
We consider the problem of fair resource allocation in a system where user demands are dynamic, that is, where user demands vary over time. Our key observation is that the classica…
PULSE: Accelerating Distributed Pointer-Traversals on Disaggregated Memory (Extended Version)
Yupeng Tang, Seung-seob Lee, Abhishek Bhattacharjee +1
Caches at CPU nodes in disaggregated memory architectures amortize the high data access latency over the network. However, such caches are fundamentally unable to improve performan…
A Multi-Site Accelerator-Rich Processing Fabric for Scalable Brain-Computer Interfacing
Karthik Sriram, Raghavendra Pradyumna Pothukuchi, Michał Gerasimiuk +5
Hull is an accelerator-rich distributed implantable Brain-Computer Interface (BCI) that reads biological neurons at data rates that are 2-3 orders of magnitude higher than the prio…
GCS: Generalized Cache Coherence For Efficient Synchronization
Yanpeng Yu, Seung-seob Lee, Anurag Khandelwal +1
We explore the design of scalable synchronization primitives for disaggregated shared memory. Porting existing synchronization primitives to disaggregated shared memory results in…
MIND: In-Network Memory Management for Disaggregated Data Centers
Seung-seob Lee, Yanpeng Yu, Yupeng Tang +3
Memory-compute disaggregation promises transparent elasticity, high utilization and balanced usage for resources in data centers by physically separating memory and compute into ne…
Cloud Programming Simplified: A Berkeley View on Serverless Computing
Eric Jonas, Johann Schleier-Smith, Vikram Sreekanti +11
Serverless cloud computing handles virtually all the system administration operations needed to make it easier for programmers to use the cloud. It provides an interface that great…