Network and Compiler Optimizations for Efficient Linear Algebra Kernels in Private Transformer Inference

Karthik Garimella, Negar Neda, Austin Ebel +2

Large language model (LLM) based services are primarily structured as client-server interactions, with clients sending queries directly to cloud providers that host LLMs. This appr…