Cascade: Token-Sharded Private LLM Inference

Rahul Thomas, Louai Zahran, Erica Choi +3

As LLMs continue to increase in parameter size, the computational resources required to run them are available to fewer parties. Therefore, third-party inference services -- where…