12 citations · 23 across the 25 of their papers we have counts for
4 papers · 1 filter
HW-Router: Hardware-Aware Routing for Scalable Multi-LLM Serving
Ahasan Kabir, Jiaqi Xue, Mengxin Zheng +1
Modern large language model (LLM) serving platforms deploy multiple models across different GPUs, requiring routers to direct incoming queries to appropriate LLMs. However, existin…
INFRAMIND: Infrastructure-Aware Multi-Agent Orchestration
Ahasan Kabir, Jiaqi Xue, Mengxin Zheng +1
Existing multi-agent LLM orchestration methods, ranging from brute-force ensembles to learned routers, select models and topologies based on task and model features. However, these…
RobPI: Robust Private Inference against Malicious Client
Jiaqi Xue, Mengxin Zheng, Qian Lou
The increased deployment of machine learning inference in various applications has sparked privacy concerns. In response, private inference (PI) protocols have been created to allo…
R2-Router: A New Paradigm for LLM Routing with Reasoning
Jiaqi Xue, Qian Lou, Jiarong Xing +1
As LLMs proliferate with diverse capabilities and costs, LLM routing has emerged by learning to predict each LLM's quality and cost for a given query, then selecting the one with h…