11 papers
When Calibration Rankings Reverse: Accuracy-Controlled Evaluation for Fair Comparison of LLMs
Zhichao Yang, Caiqi Zhang, Ruihan Yang +3
Calibration evaluates whether a model confidence aligns with its empirical accuracy. Existing studies often compare the calibration of different large language models using global…
StainFlow: Entity-Stain Tracking and Evidence Linking for Process Rewards in GUI Agents
Haojie Hao, Longkun Hao, Yihang Lou +8
Reinforcement Learning (RL) has become a promising approach for improving GUI Agents in long-horizon, stochastic digital environments, but trajectory-level success feedback is too…
Medical thinking with multiple images
Zonghai Yao, Benlu Wang, Yifan Zhang +8
Large language models perform well on many medical QA benchmarks, but real clinical reasoning often requires integrating evidence across multiple images rather than interpreting a…
RiTeK: A Dataset for Large Language Models Complex Reasoning over Textual Knowledge Graphs in Medicine
Jiatan Huang, Mingchen Li, Zonghai Yao +8
Answering complex real-world questions in the medical domain often requires accurate retrieval from medical Textual Knowledge Graphs (medical TKGs), as the relational path informat…
TARSE: Test-Time Adaptation via Retrieval of Skills and Experience for Reasoning Agents
Junda Wang, Zonghai Tao, Hansi Zeng +3
Complex clinical decision making often fails not because a model lacks facts, but because it cannot reliably select and apply the right procedural knowledge and the right prior exa…
MedQA-CS: Objective Structured Clinical Examination (OSCE)-Style Benchmark for Evaluating LLM Clinical Skills
Zonghai Yao, Zihao Zhang, Chaolong Tang +8
Artificial intelligence (AI) and large language models (LLMs) in healthcare require advanced clinical skills (CS), yet current benchmarks fail to evaluate these comprehensively. We…