1 citations · 1 across the 23 of their papers we have counts for
4 papers · 1 filter
The 1/W Law: An Analytical Study of Context-Length Routing Topology and GPU Generation Gains for LLM Inference Energy Efficiency
Huamin Chen, Xunzhuo Liu, Yuhan Liu +3
How many tokens can a GPU inference cluster deliver per watt? Across deployments of identical hardware, the answer varies by 40x -- not because of software inefficiency, but becaus…
Token-Budget-Aware Pool Routing for Cost-Efficient LLM Inference
Huamin Chen, Xunzhuo Liu, Junchen Jiang +2
Production vLLM fleets provision every instance for worst-case context length, wasting 4-8x concurrency on the 80-95% of requests that are short and simultaneously triggering KV-ca…
FleetOpt: Analytical Fleet Provisioning for LLM Inference with Compress-and-Route as Implementation Mechanism
Huamin Chen, Xunzhuo Liu, Yuhan Liu +3
Modern LLM GPU fleets are provisioned for worst-case context lengths that the vast majority of requests never approach, wasting GPU capacity on idle KV-cache slots. We present Flee…
inference-fleet-sim: A Queueing-Theory-Grounded Fleet Capacity Planner for LLM Inference
Huamin Chen, Xunzhuo Liu, Yuhan Liu +3
Sizing a GPU fleet for LLM inference is harder than it looks. The obvious questions -- how many GPUs, which type, where to split a two-pool fleet -- have no closed-form answers. Th…