Cost-Efficient LLM Serving in the Cloud: VM Selection with KV Cache Offloading

Kihyun Kim, Jinwoo Kim, Hyunsun Chung +3

LLM inference is essential for applications like text summarization, translation, and data analysis, but the high cost of GPU instances from Cloud Service Providers (CSPs) like AWS…