2 papers
cs.LG2025
Cost-Efficient LLM Serving in the Cloud: VM Selection with KV Cache Offloading
Kihyun Kim, Jinwoo Kim, Hyunsun Chung +3
LLM inference is essential for applications like text summarization, translation, and data analysis, but the high cost of GPU instances from Cloud Service Providers (CSPs) like AWS…
cs.AI2025
Shared Disk KV Cache Management for Efficient Multi-Instance Inference in RAG-Powered LLMs
Hyungwoo Lee, Kihyun Kim, Jinwoo Kim +5
Recent large language models (LLMs) face increasing inference latency as input context length and model size continue to grow. In particular, the retrieval-augmented generation (RA…