4 papers
Visible Reasoning Is Not a Universal Optimizer: Persona- and Thinking-Dependent Effects in Analytics Code Generation
Bhawani Shankar Leelar, Pawan Chorasiya, Davin Hill +2
Visible Chain-of-Thought (CoT) is often treated as a broadly useful reasoning instruction, yet analytics code generation combines natural-language ambiguity, schema grounding, targ…
MedFabric: Gold Evidence Hides the Difficulty of Word-Level Medical Fabrication Detection
Tung Sum Thomas Kwok, Qian Qian, Xiaofeng Lin +8
Large language models fabricate in medicine, producing fluent statements that are factually wrong, so reliable fabrication detection is a prerequisite for clinical deployment. Repo…
Play Favorites: A Statistical Method to Measure Self-Bias in LLM-as-a-Judge
Evangelia Spiliopoulou, Riccardo Fogliato, Hanna Burnsky +4
Large language models (LLMs) can serve as judges that offer rapid and reliable assessments of other LLM outputs. However, models may systematically assign overly favorable ratings…
Leveraging LLMs for Dialogue Quality Measurement
Jinghan Jia, Abi Komma, Timothy Leffel +5
In task-oriented conversational AI evaluation, unsupervised methods poorly correlate with human judgments, and supervised approaches lack generalization. Recent advances in large l…