KVSwap: Disk-aware KV Cache Offloading for Long-Context On-device Inference

Huawei Zhang, Chunwei Xia, Zheng Wang

Language models (LMs) underpin emerging mobile and embedded AI applications like meeting and video summarization and document analysis, which often require processing multiple long…