5 papers
MatrAIx: Simulating the World with 8.3 Billion Persona Agents
Xiaomin Li, Yuexing Hao, Jianheng Hou +90
Human evaluation of AI systems and digital products is costly, slow, and difficult to scale. Offline evaluations are more scalable but often abstract away human diversity and inter…
Mind the Delay: Latency-aware Query Routing for Heterogeneous LLM pools
Shivam Patel, Akaash R. Parthasarathy, Ankur Mallick +2
Large language model (LLM) query routers typically assign each query to a suitable model that balances response accuracy and monetary cost. Subsequently, separate system-level load…
Federate the Router: Learning Language Model Routers with Sparse and Decentralized Evaluations
Baris Askin, Shivam Patel, Anupam Nayak +4
Large language models (LLMs) are increasingly accessed as remotely hosted services by edge and enterprise clients that cannot run frontier models locally. Since models vary widely…
LOCUS: Low-Dimensional Model Embeddings for Efficient Model Exploration, Comparison, and Selection
Shivam Patel, William Cocke, Gauri Joshi
The rapidly growing ecosystem of Large Language Models (LLMs) makes it increasingly challenging to manage and utilize the vast and dynamic pool of models effectively. We propose LO…
ProxRouter: Proximity-Weighted LLM Query Routing for Improved Robustness to Outliers
Shivam Patel, Neharika Jali, Ankur Mallick +1
Large language model (LLM) query routers are critical to modern AI platforms as they seek to improve efficiency by assigning inference queries to accurate, yet low-cost models. Par…