RouterWorld: Scaling Conversational AI with Latency-Aware LLM Routing

Jinyang Li, Denis Emelin, Simone Tedeschi +6

Conversational AI systems built on Large Language Models (LLMs) require routing mechanisms that balance answer quality and response latency at scale. However, existing LLM routing…