Enabling Efficient Serverless Inference Serving for LLM (Large Language Model) in the Cloud

Himel Ghosh

This review report discusses the cold start latency in serverless inference and existing solutions. It particularly reviews the ServerlessLLM method, a system designed to address t…