Towards Resource-Efficient Serverless LLM Inference with SLINFER

Chuhao Xu, Zijun Li, Quan Chen +3

The rise of LLMs has driven demand for private serverless deployments, characterized by moderate-sized models and infrequent requests. While existing serverless solutions follow ex…