Showing 2026Show all
3 papers · 1 filter
cs.AI2026
ExecRubrics: Executable Tool-Augmented Rubrics for Verifiable and Efficient Long-Form Evaluation
Kaustubh D. Dhole, Charles L. A. Clarke, Eugene Y. Agichtein
Rubrics aim to make language-model evaluation transparent by decomposing response quality into interpretable criteria. However, natural-language rubrics are often ambiguous, requir…
cs.CL2026
How Fine-Grained Should a RAG Benchmark Be? A Hierarchical Framework for Synthetic Question Generation
Chase M. Fensore, Kaustubh Dhole, Jason Fan +2
Evaluating retrieval-augmented generation (RAG) systems requires benchmarks that capture diverse question characteristics, yet practitioners lack empirical guidance on which dimens…
cs.IR2026
RubricRAG: Towards Interpretable and Reliable LLM Evaluation via Domain Knowledge Retrieval for Rubric Generation
Kaustubh D. Dhole, Eugene Agichtein
Large language models (LLMs) are increasingly evaluated and sometimes trained using automated graders such as LLM-as-judges that output scalar scores or preferences. While convenie…