5 papers
Enabling KV Caching of Shared Prefix for Diffusion Language Models
Younghun Go, Jaehoon Han, Changyong Shin +2
Key-value (KV) caching for shared prefixes is essential for high-throughput large language model (LLM) serving, but it faces critical challenges in emerging diffusion language mode…
Unified KV Pooling to Accelerate Long-Context LLM Serving
Minchul Kang, Changyong Shin, Jinwoo Jeong +6
Long-context LLM serving requires offloading KV caches to host-memory and SSDs, but existing mechanisms are not designed for such long contexts. We observe significant inefficienci…
Training Time Prediction for Mixed Precision-based Distributed Training
Minchul Kang, Changyong Shin, Jinwoo Jeong +5
Accurate prediction of training time in distributed deep learning is crucial for resource allocation, cost estimation, and job scheduling. We observe that the floating-point precis…
GPU Memory Prediction for Multimodal Model Training
Jinwoo Jeong, Minchul Kang, Younghun Go +5
As deep learning models in agentic AI systems grow in scale and complexity, GPU memory requirements increase and often exceed the available GPU memory capacity, so that out-of-memo…
Prediction of Permissioned Blockchain Performance for Resource Scaling Configurations
Seungwoo Jung, Yeonho Yoo, Gyeongsik Yang +1
Blockchain is increasingly offered as blockchain-as-a-service (BaaS) by cloud service providers. However, configuring BaaS appropriately for optimal performance and reliability res…