1 citations · 2 across the 11 of their papers we have counts for
4 papers · 1 filter
Routing Without Embeddings: Fast And Interpretable Routing With Regular Expressions
Yifan Lu, Qiyue Zhang, Haotian Shan +2
Large Language Model (LLM) routers commonly rely on neural query embeddings, with larger encoders expected to better capture query intent and difficulty. Yet scaling Qwen2.5 encode…
The Routing Plateau: Understanding the Accuracy Limits of LLM Routers
Yifan Lu, Qiyue Zhang, Shenrun Zhang +4
LLM routing has become a popular approach to improve the cost-quality trade-off of LLM services by adaptively selecting a model for each query. Recent work has explored a broad ran…
S*: Test Time Scaling for Code Generation
Dacheng Li, Shiyi Cao, Chengkun Cao +6
Increasing test-time compute for LLMs shows promise across domains but remains underexplored in code generation, despite extensive study in math. In this paper, we propose S*, the…
BlendServe: Optimizing Offline Inference for Auto-regressive Large Models with Resource-aware Batching
Yilong Zhao, Shuo Yang, Kan Zhu +5
Offline batch inference, which leverages the flexibility of request batching to achieve higher throughput and lower costs, is becoming more popular for latency-insensitive applicat…