A First Look At Efficient And Secure On-Device LLM Inference Against KV Leakage

Huan Yang, Deyu Zhang, Yudong Zhao +2

Running LLMs on end devices has garnered significant attention recently due to their advantages in privacy preservation. With the advent of lightweight LLM models and specially des…