3 papers
cs.LG2025
Cost-Efficient LLM Serving in the Cloud: VM Selection with KV Cache Offloading
Kihyun Kim, Jinwoo Kim, Hyunsun Chung +3
LLM inference is essential for applications like text summarization, translation, and data analysis, but the high cost of GPU instances from Cloud Service Providers (CSPs) like AWS…
cs.AI2025
Shared Disk KV Cache Management for Efficient Multi-Instance Inference in RAG-Powered LLMs
Hyungwoo Lee, Kihyun Kim, Jinwoo Kim +5
Recent large language models (LLMs) face increasing inference latency as input context length and model size continue to grow. In particular, the retrieval-augmented generation (RA…
cs.LG2024
MPruner: Optimizing Neural Network Size with CKA-Based Mutual Information Pruning
Seungbeom Hu, ChanJun Park, Andrew Ferraiuolo +4
Determining the optimal size of a neural network is critical, as it directly impacts runtime performance and memory usage. Pruning is a well-established model compression technique…