3 papers
cs.PF2026
SALT: Salience-Aware Lexical Trie for Long-Context Compression
Oteo Mamo, Hyunjin Yi, Joydhriti Choudhury +2
As large language models (LLMs) process increasingly longer prompts, computation and KV-cache memory costs have emerged as major bottlenecks in inference systems. Existing input-le…
cs.DC2026
DUAL-BLADE: Dual-Path NVMe-Direct KV-Cache Offloading for Edge LLM Inference
Bodon Jeong, Hongsu Byun, Youngjae Kim +4
The increasing deployment of Large Language Model (LLM) inference on edge AI systems demands efficient execution under tight memory budgets. A key challenge arises from Key-Value (…
cs.AR2026
Comparative Characterization of KV Cache Management Strategies for LLM Inference
Oteo Mamo, Olga Kogiou, Hyunjin Yi +1
Efficient inference with Large Language Models (LLMs) increasingly relies on Key-Value (KV) caches to store previously computed key and value vectors at each layer. These caches ar…