3 papers
cs.AI2025
IRT-Router: Effective and Interpretable Multi-LLM Routing via Item Response Theory
Wei Song, Zhenya Huang, Cheng Cheng +5
Large language models (LLMs) have demonstrated exceptional performance across a wide range of natural language tasks. However, selecting the optimal LLM to respond to a user query…
cs.HC2024
A Dataset for the Validation of Truth Inference Algorithms Suitable for Online Deployment
Fei Wang, Haoyu Liu, Haoyang Bi +9
For the purpose of efficient and cost-effective large-scale data labeling, crowdsourcing is increasingly being utilized. To guarantee the quality of data labeling, multiple annotat…
cs.HC2023
Towards Long-term Annotators: A Supervised Label Aggregation Baseline
Haoyu Liu, Fei Wang, Minmin Lin +6
Relying on crowdsourced workers, data crowdsourcing platforms are able to efficiently provide vast amounts of labeled data. Due to the variability in the annotation quality of crow…