4 papers
Unified KV Pooling to Accelerate Long-Context LLM Serving
Minchul Kang, Changyong Shin, Jinwoo Jeong +6
Long-context LLM serving requires offloading KV caches to host-memory and SSDs, but existing mechanisms are not designed for such long contexts. We observe significant inefficienci…
Enabling KV Caching of Shared Prefix for Diffusion Language Models
Younghun Go, Jaehoon Han, Changyong Shin +2
Key-value (KV) caching for shared prefixes is essential for high-throughput large language model (LLM) serving, but it faces critical challenges in emerging diffusion language mode…
Training Time Prediction for Mixed Precision-based Distributed Training
Minchul Kang, Changyong Shin, Jinwoo Jeong +5
Accurate prediction of training time in distributed deep learning is crucial for resource allocation, cost estimation, and job scheduling. We observe that the floating-point precis…
GPU Memory Prediction for Multimodal Model Training
Jinwoo Jeong, Minchul Kang, Younghun Go +5
As deep learning models in agentic AI systems grow in scale and complexity, GPU memory requirements increase and often exceed the available GPU memory capacity, so that out-of-memo…