Raincheck: Deferred Multimodal Prefill for Efficient Multi-Turn MLLM Inference

Wenjie Huang, Jun Zhang, Hongyaoxing Gu +10

Multi-turn conversations with multimodal large language models (MLLMs) reuse input context, but long multimodal inputs increase time to first token (TTFT). Query-aware hard pruning…