11 papers
HW-Router: Hardware-Aware Routing for Scalable Multi-LLM Serving
Ahasan Kabir, Jiaqi Xue, Mengxin Zheng +1
Modern large language model (LLM) serving platforms deploy multiple models across different GPUs, requiring routers to direct incoming queries to appropriate LLMs. However, existin…
INFRAMIND: Infrastructure-Aware Multi-Agent Orchestration
Ahasan Kabir, Jiaqi Xue, Mengxin Zheng +1
Existing multi-agent LLM orchestration methods, ranging from brute-force ensembles to learned routers, select models and topologies based on task and model features. However, these…
R2-Router: A New Paradigm for LLM Routing with Reasoning
Jiaqi Xue, Qian Lou, Jiarong Xing +1
As LLMs proliferate with diverse capabilities and costs, LLM routing has emerged by learning to predict each LLM's quality and cost for a given query, then selecting the one with h…
RobPI: Robust Private Inference against Malicious Client
Jiaqi Xue, Mengxin Zheng, Qian Lou
The increased deployment of machine learning inference in various applications has sparked privacy concerns. In response, private inference (PI) protocols have been created to allo…
SoK: Can Fully Homomorphic Encryption Support General AI Computation? A Functional and Cost Analysis
Jiaqi Xue, Xin Xin, Wei Zhang +11
Artificial intelligence (AI) increasingly powers sensitive applications in domains such as healthcare and finance, relying on both linear operations (e.g., matrix multiplications i…
PRO: Enabling Precise and Robust Text Watermark for Open-Source LLMs
Jiaqi Xue, Yifei Zhao, Mansour Al Ghanim +4
Text watermarking for large language models (LLMs) enables model owners to verify text origin and protect intellectual property. While watermarking methods for closed-source LLMs a…