11 citations · 24 across the 26 of their papers we have counts for
3 papers · 1 filter
Oneiros: KV Cache Optimization through Parameter Remapping for Multi-tenant LLM Serving
Ruihao Li, Shagnik Pal, Vineeth Narayan Pullu +4
KV cache accelerates LLM inference by avoiding redundant computation, at the expense of memory. To support larger KV caches, prior work extends GPU memory with CPU memory via CPU-o…
SOL: Safe On-Node Learning in Cloud Platforms
Yawen Wang, Daniel Crankshaw, Neeraja J. Yadwadkar +3
Cloud platforms run many software agents on each server node. These agents manage all aspects of node operation, and in some cases frequently collect data and make decisions. Unfor…
Cloud Programming Simplified: A Berkeley View on Serverless Computing
Eric Jonas, Johann Schleier-Smith, Vikram Sreekanti +11
Serverless cloud computing handles virtually all the system administration operations needed to make it easier for programmers to use the cloud. It provides an interface that great…