MPCache: MPC-Friendly KV Cache Eviction for Efficient Private LLM Inference

Wenxuan Zeng, Ye Dong, Jinjin Zhou +5

Private large language model (LLM) inference based on secure multi-party computation (MPC) achieves formal data privacy protection but suffers from significant latency overhead, es…