Lynx: Progressive Speculative Quantization for accelerating KV Transfer in Long-Context Inference

Wenchen Han, Gingfung Matthew Yeung, Marco Barletta +3

Long-context inference is increasingly common in large language model (LLM) serving, driven by retrieval-augmented generation and agentic systems. In disaggregated inference, these…