2 papers
cs.PF2026
SALT: Salience-Aware Lexical Trie for Long-Context Compression
Oteo Mamo, Hyunjin Yi, Joydhriti Choudhury +2
As large language models (LLMs) process increasingly longer prompts, computation and KV-cache memory costs have emerged as major bottlenecks in inference systems. Existing input-le…
cs.AR2026
Comparative Characterization of KV Cache Management Strategies for LLM Inference
Oteo Mamo, Olga Kogiou, Hyunjin Yi +1
Efficient inference with Large Language Models (LLMs) increasingly relies on Key-Value (KV) caches to store previously computed key and value vectors at each layer. These caches ar…