4 papers
TraCT: Disaggregated LLM Serving with CXL Shared Memory KV Cache at Rack-Scale
Dongha Yoon, Younghoon Min, Hoshik Kim +2
Disaggregated LLM serving improves resource efficiency by separating the compute-intensive prefill phase from the latency-critical decode phase. However, this architecture introduc…
Accelerating Sparse Matrix-Matrix Multiplication on GPUs with Processing Near HBMs
Shiju Li, Younghoon Min, Hane Yie +5
Sparse General Matrix-Matrix Multiplication (SpGEMM) is a fundamental operation in numerous scientific computing and data analytics applications, often bottlenecked by irregular me…
MoSKA: Mixture of Shared KV Attention for Efficient Long-Sequence LLM Inference
Myunghyun Rhee, Sookyung Choi, Euiseok Kim +3
The escalating context length in Large Language Models (LLMs) creates a severe performance bottleneck around the Key-Value (KV) cache, whose memory-bound nature leads to significan…
cMPI: Using CXL Memory Sharing for MPI One-Sided and Two-Sided Inter-Node Communications
Xi Wang, Bin Ma, Jongryool Kim +3
Message Passing Interface (MPI) is a foundational programming model for high-performance computing. MPI libraries traditionally employ network interconnects (e.g., Ethernet and Inf…