1 citations · 1 across the 14 of their papers we have counts for
16 papers
The 1/W Law: An Analytical Study of Context-Length Routing Topology and GPU Generation Gains for LLM Inference Energy Efficiency
Huamin Chen, Xunzhuo Liu, Yuhan Liu +3
How many tokens can a GPU inference cluster deliver per watt? Across deployments of identical hardware, the answer varies by 40x -- not because of software inefficiency, but becaus…
Token-Budget-Aware Pool Routing for Cost-Efficient LLM Inference
Huamin Chen, Xunzhuo Liu, Junchen Jiang +2
Production vLLM fleets provision every instance for worst-case context length, wasting 4-8x concurrency on the 80-95% of requests that are short and simultaneously triggering KV-ca…
Dual-Pool Token-Budget Routing for Cost-Efficient and Reliable LLM Serving
Xunzhuo Liu, Bowei He, Xue Liu +3
Production vLLM fleets typically provision each instance for the worst-case context length, leading to substantial KV-cache over-allocation and under-utilized concurrency. In pract…
The Workload-Router-Pool Architecture for LLM Inference Optimization: A Vision Paper from the vLLM Semantic Router Project
Huamin Chen, Xunzhuo Liu, Bowei He +5
Over the past year, the vLLM Semantic Router project has released a series of work spanning: (1) core routing mechanisms -- signal-driven routing, context-length pool routing, rout…
From Inference Routing to Agent Orchestration: Declarative Policy Compilation with Cross-Layer Verification
Huamin Chen, Xunzhuo Liu, Bowei He +1
The Semantic Router DSL is a non-Turing-complete policy language deployed in production for per-request LLM inference routing: content signals (embedding similarity, PII detection,…
Knowledge Access Beats Model Size: Memory Augmented Routing for Persistent AI Agents
Xunzhuo Liu, Bowei He, Xue Liu +3
Production AI agents frequently receive user-specific queries that are highly repetitive, with up to 47\% being semantically similar to prior interactions, yet each query is typica…