11 papers
LLMRouterBench: A Massive Benchmark and Unified Framework for LLM Routing
Hao Li, Yiqun Zhang, Zhaoyan Guo +9
Large language model (LLM) routing assigns each query to the most suitable model from an ensemble. We introduce LLMRouterBench, a large-scale benchmark and unified framework for LL…
How Real is Your Jailbreak? Fine-grained Jailbreak Evaluation with Anchored Reference
Songyang Liu, Chaozhuo Li, Rui Pu +5
Jailbreak attacks present a significant challenge to the safety of Large Language Models (LLMs), yet current automated evaluation methods largely rely on coarse classifications tha…
LANCET: Neural Intervention via Structural Entropy for Mitigating Faithfulness Hallucinations in LLMs
Chenxu Wang, Chaozhuo Li, Pengbo Wang +7
Large Language Models have revolutionized information processing, yet their reliability is severely compromised by faithfulness hallucinations. While current approaches attempt to…
ICL-Router: In-Context Learned Model Representations for LLM Routing
Chenxu Wang, Hao Li, Yiqun Zhang +6
Large language models (LLMs) often exhibit complementary strengths. Model routing harnesses these strengths by dynamically directing each query to the most suitable model, given a…
Learning Compact Representations of LLM Abilities via Item Response Theory
Jianhao Chen, Chenxu Wang, Gengrui Zhang +5
Recent years have witnessed a surge in the number of large language models (LLMs), yet efficiently managing and utilizing these vast resources remains a significant challenge. In t…
PerPilot: Personalizing VLM-based Mobile Agents via Memory and Exploration
Xin Wang, Zhiyao Cui, Hao Li +10
Vision language model (VLM)-based mobile agents show great potential for assisting users in performing instruction-driven tasks. However, these agents typically struggle with perso…