$A^3$: Attention-Aware Accurate KV Cache Fusion for Fast Large Language Model Serving

Yuechi Zhou, Yi Su, Jianxin Zhang +5

Large language models (LLMs) have demonstrated strong capabilities in processing long contexts, enabling them to tackle tasks involving long textual inputs such as multi-turn conve…