PrefixWall: Mitigating Prefix Caching Side Channels in Shared LLM Systems

Panagiotis Georgios Pennas, Konstantinos Papaioannou, Marco Guarnieri +1

Large Language Models (LLMs) rely on optimizations like Automatic Prefix Caching (APC) to accelerate inference. APC works by reusing previously computed states for the beginning pa…