Comet: Accelerating Private Inference for Large Language Model by Predicting Activation Sparsity

Guang Yan, Yuhui Zhang, Zimu Guo +6

With the growing use of large language models (LLMs) hosted on cloud platforms to offer inference services, privacy concerns about the potential leakage of sensitive information ar…