C3: Speech-Guided KV Retrieval for Long-Context SpeechLLM Inference

Dancheng Liu, Markus Müller, Rupak Vignesh Swaminathan +3

Speech language models encode audio at a much higher rate than text, causing the KV cache to dominate the memory, bandwidth, and decoding cost of long-form generation. KV retrieval…