most citedDroidSpeak: KV Cache Sharing for Cross-LLM Communication and Multi-LLM Serving

2 citations · 2 across the 2 of their papers we have counts for

collaborators
Showing cs.LGShow all

1 paper · 1 filter

cs.LG2024

CacheBlend: Fast Large Language Model Serving for RAG with Cached Knowledge Fusion

Jiayi Yao, Hanchen Li, Yuhan Liu +6

Large language models (LLMs) often incorporate multiple text chunks in their inputs to provide the necessary contexts. To speed up the prefill of the long LLM inputs, one can pre-c…