56 citations · 73 across the 8 of their papers we have counts for
3 papers · 1 filter
Prompt Cache: Modular Attention Reuse for Low-Latency Inference
In Gim, Guojun Chen, Seung-seob Lee +3
We present Prompt Cache, an approach for accelerating inference for large language models (LLM) by reusing attention states across different LLM prompts. Many input prompts have ov…
PULSE: Accelerating Distributed Pointer-Traversals on Disaggregated Memory (Extended Version)
Yupeng Tang, Seung-seob Lee, Abhishek Bhattacharjee +1
Caches at CPU nodes in disaggregated memory architectures amortize the high data access latency over the network. However, such caches are fundamentally unable to improve performan…
GCS: Generalized Cache Coherence For Efficient Synchronization
Yanpeng Yu, Seung-seob Lee, Anurag Khandelwal +1
We explore the design of scalable synchronization primitives for disaggregated shared memory. Porting existing synchronization primitives to disaggregated shared memory results in…