10 papers
SubtleMemory: A Benchmark for Fine-Grained Relational Memory Discrimination in Long-Horizon AI Agents
Wenxuan Wang, Haoyu Sun, Fukuan Hou +4
Persistent AI assistants, such as OpenClaw, accumulate large collections of related memories over long-term interactions. As these memories grow, they may reinforce one another, di…
Agent Planning Benchmark: A Diagnostic Framework for Planning Capabilities in LLM Agents
Haoyu Sun, Wenxuan Wang, Mingyang Song +5
Planning is central to LLM agents: before acting, an agent must decompose goals, select tools, reason over constraints, and decide when a task is infeasible. Yet existing agent eva…
Beyond the Leaderboard: Rethinking Medical Benchmarks for Large Language Models
Wenting Chen, Guo Yu, Yiu-Fai Cheung +5
Large language models (LLMs) show significant potential in healthcare, prompting numerous benchmarks to evaluate their capabilities. However, concerns persist regarding the reliabi…
Toward Personalized LLM-Powered Agents: Foundations, Evaluation, and Future Directions
Yue Xu, Qian Chen, Zizhan Ma +5
Large language models have enabled agentic systems that reason, plan, and interact with tools and environments to accomplish complex tasks. As these agents operate over extended in…
MMedExpert-R1: Strengthening Multimodal Medical Reasoning via Domain-Specific Adaptation and Clinical Guideline Reinforcement
Meidan Ding, Jipeng Zhang, Wenxuan Wang +4
Medical Vision-Language Models (MedVLMs) excel at perception tasks but struggle with complex clinical reasoning required in real-world scenarios. While reinforcement learning (RL)…
MedEinst: Benchmarking the Einstellung Effect in Medical LLMs through Counterfactual Differential Diagnosis
Wenting Chen, Zhongrui Zhu, Guolin Huang +1
Despite achieving high accuracy on medical benchmarks, LLMs exhibit the Einstellung Effect in clinical diagnosis--relying on statistical shortcuts rather than patient-specific evid…