2 citations · 3 across the 6 of their papers we have counts for
5 papers · 1 filter
JudgeMoE: Distributional Aggregation for LLM-as-a-Judge
Yiqi Liu, Joseph James, Yang Wang +3
When an LLM judge scores an output, its score distribution retains uncertainty and disagreement information that is lost after scalar compression. We introduce JudgeMoE, a lightwei…
When Tool-Backed Skill Retrieval Fails: Source-Style Collapse in Executable Capability Retrieval
Yiqi Liu, Joseph James, Yang Wang +2
Large-scale agents increasingly rely on retrieval to access external capabilities. We study this retrieval gate in structured tools and APIs, a measurable class of tool-backed exec…
Counting on Consensus: Selecting the Right Inter-annotator Agreement Metric for NLP Annotation and Evaluation
Joseph James
Human annotation remains the foundation of reliable and interpretable data in Natural Language Processing (NLP). As annotation and evaluation tasks continue to expand, from categor…
Evaluating LLM-Based Grant Proposal Review via Structured Perturbations
William Thorne, Joseph James, Yang Wang +2
As AI-assisted grant proposals outpace manual review capacity in a kind of ``Malthusian trap'' for the research ecosystem, this paper investigates the capabilities and limitations…
RIGOURATE: Quantifying Scientific Exaggeration with Evidence-Aligned Claim Evaluation
Joseph James, Chenghao Xiao, Yucheng Li +2
Scientific rigour tends to be sidelined in favour of bold statements, leading authors to overstate claims beyond what their results support. We present RIGOURATE, a two-stage multi…