4 papers
Bridging LLM Serving and CXL-SSDs with Chunk-Aware KV Cache Management
Hyunsun Chung, Taewan Noh, Minji Kim +3
NAND-backed storage offers the capacity needed to scale LLM prefix caching, but its block I/O path incurs CPU cache contention and host-DRAM staging in addition to NAND latency. Ou…
OpenCXD: An Open Real-Device-Guided Hybrid Evaluation Framework for CXL-SSDs
Hyunsun Chung, Junhyeok Park, Taewan Noh +4
The advent of Compute Express Link (CXL) enables SSDs to participate in the memory hierarchy as large-capacity, byte-addressable memory devices. These CXL-enabled SSDs (CXL-SSDs) o…
Cost-Efficient LLM Serving in the Cloud: VM Selection with KV Cache Offloading
Kihyun Kim, Jinwoo Kim, Hyunsun Chung +3
LLM inference is essential for applications like text summarization, translation, and data analysis, but the high cost of GPU instances from Cloud Service Providers (CSPs) like AWS…
A Host-SSD Collaborative Write Accelerator for LSM-Tree-Based Key-Value Stores
KiHwan Kim, Hyunsun Chung, Seonghoon Ahn +6
Log-Structured Merge (LSM) tree-based Key-Value Stores (KVSs) are widely adopted for their high performance in write-intensive environments, but they often face performance degrada…