9 papers
RouteJudge: An Open Platform for Reproducible and Preference-Aware LLM Routing
Guannan Lai, Haoran Hu, Han-Jia Ye
We present RouteJudge, an online pairwise preference evaluation framework for LLM routing systems, with a public platform available at https://routejudge.cn. Different from model-l…
From Sampled Outcomes to Capability Distributions: Rethinking Supervision for LLM Routing
Guannan Lai, Haoran Hu, Long Chen +2
Existing LLM routing methods typically treat a model's single response to a query as its capability label for training routers. However, because LLM generation is inherently stocha…
The Golden Subspace: Where Efficiency Meets Generalization in Continual Test-Time Adaptation
Guannan Lai, Da-Wei Zhou, Zhenguo Li +1
Continual Test-Time Adaptation (CTTA) aims to enable models to adapt online to unlabeled data streams under distribution shift without accessing source data. Existing CTTA methods…
The Lie of the Average: How Class Incremental Learning Evaluation Deceives You?
Guannan Lai, Da-Wei Zhou, Xin Yang +1
Class Incremental Learning (CIL) requires models to continuously learn new classes without forgetting previously learned ones, while maintaining stable performance across all possi…
When Routing Collapses: On the Degenerate Convergence of LLM Routers
Guannan Lai, Han-Jia Ye
LLM routing aims to achieve a favorable quality--cost trade-off by dynamically assigning easy queries to smaller models and harder queries to stronger ones. However, across both un…
MMR-Bench: A Comprehensive Benchmark for Multimodal LLM Routing
Haoxuan Ma, Guannan Lai, Han-Jia Ye
Multimodal large language models (MLLMs) have advanced rapidly, yet heterogeneity in architecture, alignment strategies, and efficiency means that no single model is uniformly supe…