7 papers
vLLM Semantic Router: Signal Driven Decision Routing for Mixture-of-Modality Models
Xunzhuo Liu, Huamin Chen, Samzong Lu +30
As large language models (LLMs) diversify across modalities, capabilities, and cost profiles, the problem of intelligent request routing: selecting the right model for each query a…
Dual-Pool Token-Budget Routing for Cost-Efficient and Reliable LLM Serving
Xunzhuo Liu, Bowei He, Xue Liu +3
Production vLLM fleets typically provision each instance for the worst-case context length, leading to substantial KV-cache over-allocation and under-utilized concurrency. In pract…
Knowledge Access Beats Model Size: Memory Augmented Routing for Persistent AI Agents
Xunzhuo Liu, Bowei He, Xue Liu +3
Production AI agents frequently receive user-specific queries that are highly repetitive, with up to 47\% being semantically similar to prior interactions, yet each query is typica…
Visual Confused Deputy: Exploiting and Defending Perception Failures in Computer-Using Agents
Xunzhuo Liu, Bowei He, Xue Liu +3
Computer-using agents (CUAs) act directly on graphical user interfaces, yet their perception of the screen is often unreliable. Existing work largely treats these failures as perfo…
Adaptive Vision-Language Model Routing for Computer Use Agents
Xunzhuo Liu, Bowei He, Xue Liu +3
Computer Use Agents (CUAs) translate natural-language instructions into Graphical User Interface (GUI) actions such as clicks, keystrokes, and scrolls by relying on a Vision-Langua…
98 Faster LLM Routing Without a Dedicated GPU: Flash Attention, Prompt Compression, and Near-Streaming for the vLLM Semantic Router
Xunzhuo Liu, Bowei He, Xue Liu +3
System-level routers that intercept LLM requests for safety classification, domain routing, and PII detection must be both fast and operationally lightweight: they should add minim…