13 papers · 1 filter
InnoEval: On Research Idea Evaluation as a Knowledge-Grounded, Multi-Perspective Reasoning Problem
Shuofei Qiao, Yunxiang Wei, Xuehai Wang +10
The rapid evolution of Large Language Models has catalyzed a surge in scientific idea production, yet this leap has not been accompanied by a matching advance in idea evaluation. T…
Learning an Efficient Multi-Turn Dialogue Evaluator from Multiple LLM Judges
Yuqi Tang, Kehua Feng, Yunfeng Wang +6
Evaluating the conversational abilities of large language models (LLMs) remains a challenging task. Current mainstream approaches primarily rely on the "LLM-as-a-judge" paradigm, w…
Evaluating Reward Model Generalization via Pairwise Maximum Discrepancy Competitions
Shunyang Luo, Peibei Cao, Zhihui Zhu +3
Reward models (RMs) are central to aligning large language models, yet their practical effectiveness hinges on generalization to unseen prompts and shifting distributions. Most exi…
ClinDEF: A Dynamic Evaluation Framework for Large Language Models in Clinical Reasoning
Yuqi Tang, Jing Yu, Zichang Su +7
Clinical diagnosis begins with doctor-patient interaction, during which physicians iteratively gather information, determine examination and refine differential diagnosis through p…
CoT-Evo: Evolutionary Distillation of Chain-of-Thought for Scientific Reasoning
Kehua Feng, Keyan Ding, Zhihui Zhu +3
While chain-of-thought (CoT) distillation from advanced large language models (LLMs) has proven effective in general reasoning tasks, it struggles in scientific domains where even…
SAFER: Advancing Safety Alignment via Efficient Ex-Ante Reasoning
Kehua Feng, Keyan Ding, Yuhao Wang +5
Recent advancements in large language models (LLMs) have accelerated progress toward artificial general intelligence, yet their potential to generate harmful content poses critical…