Cost-Optimal LLM Routing with Limited User Feedback under User Satisfaction Guarantees

Herbert Woisetschläger, Arastun Mammadli, Ryan Zhang +1

Inference costs for large language model (LLM) applications are rapidly growing, driven by surging demand and rising infrastructure cost. Users expect high-quality responses, and i…