Cachemir: Fully Homomorphic Encrypted Inference of Generative Large Language Model with KV Cache

Ye Yu, Yifan Zhou, Yi Chen +3

Generative large language models (LLMs) have revolutionized multiple domains. Modern LLMs predominantly rely on an autoregressive decoding strategy, which generates output tokens s…