RAC: Reference-Aware Activation Compression for Communication-Efficient Split LLM Inference

Guotao Yang, Mingxi Zhao, Haopeng Li +4

Large language model (LLM) agents repeatedly process long, privacy-sensitive contexts, while cloud-only deployment exposes user data beyond the trusted endpoint and fully local dep…