56 citations · 73 across the 8 of their papers we have counts for
8 papers
Pie: A Programmable Serving System for Emerging LLM Applications
In Gim, Zhiyao Ma, Seung-seob Lee +1
Emerging large language model (LLM) applications involve diverse reasoning strategies and agentic workflows, straining the capabilities of existing serving systems built on a monol…
Asynchronous LLM Function Calling
In Gim, Seung-seob Lee, Lin Zhong
Large language models (LLMs) use function calls to interface with external tools and data source. However, the current approach to LLM function calling is inherently synchronous, w…
Blindfold: Confidential Memory Management by Untrusted Operating System
Caihua Li, Seung-seob Lee, Lin Zhong
Confidential Computing (CC) has received increasing attention in recent years as a mechanism to protect user data from untrusted operating systems (OSes). Existing CC solutions hid…
Prompt Cache: Modular Attention Reuse for Low-Latency Inference
In Gim, Guojun Chen, Seung-seob Lee +3
We present Prompt Cache, an approach for accelerating inference for large language models (LLM) by reusing attention states across different LLM prompts. Many input prompts have ov…
PULSE: Accelerating Distributed Pointer-Traversals on Disaggregated Memory (Extended Version)
Yupeng Tang, Seung-seob Lee, Abhishek Bhattacharjee +1
Caches at CPU nodes in disaggregated memory architectures amortize the high data access latency over the network. However, such caches are fundamentally unable to improve performan…
GCS: Generalized Cache Coherence For Efficient Synchronization
Yanpeng Yu, Seung-seob Lee, Anurag Khandelwal +1
We explore the design of scalable synchronization primitives for disaggregated shared memory. Porting existing synchronization primitives to disaggregated shared memory results in…
MProtect: Operating System Memory Management without Access
Caihua Li, Seung-seob Lee, Min Hong Yun +1
Modern operating systems (OSes) have unfettered access to application data, assuming that applications trust them. This assumption, however, is problematic under many scenarios whe…
MIND: In-Network Memory Management for Disaggregated Data Centers
Seung-seob Lee, Yanpeng Yu, Yupeng Tang +3
Memory-compute disaggregation promises transparent elasticity, high utilization and balanced usage for resources in data centers by physically separating memory and compute into ne…