BlockLLM: Multi-tenant Finer-grained Serving for Large Language Models

Bodun Hu, Jiamin Li, Le Xu +5

The increasing demand for Large Language Models (LLMs) across various applications has led to a significant shift in the design of deep learning serving systems. Deploying LLMs, pa…