2 papers
cs.AI2026
Beyond Accuracy and Cost: Latency-Aware LLM Query Routing for Dynamic Workloads
Shivam Patel, Akaash R. Parthasarathy, Ankur Mallick +1
Modern language query routers improve inference efficiency by assigning each query to a model that balances response quality and monetary cost. However, current query routers are l…
cs.LG2025
ProxRouter: Proximity-Weighted LLM Query Routing for Improved Robustness to Outliers
Shivam Patel, Neharika Jali, Ankur Mallick +1
Large language model (LLM) query routers are critical to modern AI platforms as they seek to improve efficiency by assigning inference queries to accurate, yet low-cost models. Par…