Performance Characterization of Expert Router for Scalable LLM Inference

Josef Pichlmeier, Philipp Ross, Andre Luckow

Large Language Models (LLMs) have experienced widespread adoption across scientific and industrial domains due to their versatility and utility for diverse tasks. Nevertheless, dep…