1 citations · 1 across the 13 of their papers we have counts for
13 papers
xDailyBench: Benchmarking LLMs on Professional Consultation for Real-Life Problems
Yongchang Peng, Qingshui Gu, Liya Zhu +31
Large language models (LLMs) are increasingly used for everyday assistance, yet existing benchmarks only partially reflect the requests users naturally make in practice. Real-world…
StartupBench: Benchmarking General-Purpose Agents on Market-Validated End-to-End Workflows
Liya Zhu, Xin Ma, Tao Liu +35
Recent advances in Large Language Models(LLMs) and agents have substantially improved the ability of AI systems to execute complex tasks. Yet existing benchmarks largely rely on re…
CoRA: Confidence-Rationale Alignment for Reliable Chain-of-Thought Reasoning
Juming Xiong, Weixin Liu, Kevin Guo +9
Chain-of-thought (CoT) reasoning can improve LLM performance, but high answer confidence may be misleading when the accompanying CoT rationale is plausible yet incomplete or poorly…
It's Not Always Sycophancy: Measuring LLM Conformity as a Function of Epistemic Uncertainty
Kevin H. Guo, Chao Yan, Avinash Baidya +5
Large language models (LLMs) are known to abandon their initial stance to conform to user pushback. While prior research largely attributes this behavior to sycophancy learned duri…
CLEAR: Revealing How Noise and Ambiguity Degrade Reliability in LLMs for Medicine
Kevin H. Guo, Chao Yan, Avinash Baidya +5
Medical large language model (LLM) evaluations rely on simplified, exam-style benchmarks that rarely reflect the ambiguity of real-world medical inquiries. We introduce the CLinica…
Stop Listening to Me! How Multi-turn Conversations Can Degrade LLM Reliability
Kevin H. Guo, Chao Yan, Avinash Baidya +5
Large language models (LLMs) excel on static benchmarks, but their performance across multi-turn conversations, which better reflect real-world usage, remains understudied. Address…
Learning When to Sample: Confidence-Aware Selective Sampling for Efficient Chain-of-Thought Reasoning
Juming Xiong, Kevin Guo, Congning Ni +7
Large language models (LLMs) can achieve strong reasoning performance through chain-of-thought (CoT) reasoning, yet they often generate unnecessarily long reasoning paths that incu…
NL2Repo-Bench: Towards Long-Horizon Repository Generation Evaluation of Coding Agents
Jingzhe Ding, Shengda Long, Changxin Pu +46
Recent advances in coding agents suggest rapid progress toward autonomous software development, yet existing benchmarks fail to rigorously evaluate the long-horizon capabilities re…
DiscoX: Benchmarking Discourse-Level Translation task in Expert Domains
Xiying Zhao, Zhoufutu Wen, Zhixuan Chen +20
The evaluation of discourse-level translation in expert domains remains inadequate, despite its centrality to knowledge dissemination and cross-lingual scholarly communication. Whi…
Multi-Agent Medical Decision Consensus Matrix System: An Intelligent Collaborative Framework for Oncology MDT Consultations
Xudong Han, Xianglun Gao, Xiaoyi Qu +1
Multidisciplinary team (MDT) consultations are the gold standard for cancer care decision-making, yet current practice lacks structured mechanisms for quantifying consensus and ens…
LPFQA: A Long-Tail Professional Forum-based Benchmark for LLM Evaluation
Liya Zhu, Peizhuang Cong, Jingzhe Ding +17
Large Language Models (LLMs) perform well on standard reasoning and question-answering benchmarks, yet such evaluations often fail to capture their ability to handle long-tail, exp…
FinSearchComp: Towards a Realistic, Expert-Level Evaluation of Financial Search and Reasoning
Liang Hu, Jianpeng Jiao, Jiashuo Liu +20
Search has emerged as core infrastructure for LLM-based agents and is widely viewed as critical on the path toward more general intelligence. Finance is a particularly demanding pr…
FutureX: An Advanced Live Benchmark for LLM Agents in Future Prediction
Zhiyuan Zeng, Jiashuo Liu, Siyuan Chen +28
Future prediction is a complex task for LLM agents, requiring a high level of analytical thinking, information gathering, contextual understanding, and decision-making under uncert…