4 papers · 1 filter
Dynamic HBM Repartitioning for Multi-Turn MoE Serving
Jinpyo Kim, Mingi Kwon, Younghoon Min +2
Long-running multi-turn requests accumulate reusable key-value (KV) state. Once this state exceeds a fixed GPU KV-cache allocation, serving systems evict reusable prefixes, repeat…
ITME: Inference Tiered Memory Expansion with Disaggregated CXL-Hybrid Memories
Hakbeom Jang, Younghoon Min, Sunwoong Kim +5
The rapid shift toward agentic and long-context workloads in Large Language Models (LLMs) is pushing the industry beyond the capacity of individual servers toward disaggregated sha…
TraCT: Disaggregated LLM Serving with CXL Shared Memory KV Cache at Rack-Scale
Dongha Yoon, Younghoon Min, Hoshik Kim +2
Disaggregated LLM serving improves resource efficiency by separating the compute-intensive prefill phase from the latency-critical decode phase. However, this architecture introduc…
Accelerating Sparse Matrix-Matrix Multiplication on GPUs with Processing Near HBMs
Shiju Li, Younghoon Min, Hane Yie +5
Sparse General Matrix-Matrix Multiplication (SpGEMM) is a fundamental operation in numerous scientific computing and data analytics applications, often bottlenecked by irregular me…