8 papers
RIMRULE: Improving Tool-Using Language Agents via MDL-Guided Rule Learning
Xiang Gao, Yuguang Yao, Qi Zhang +5
Large language models (LLMs) often struggle to use tools reliably in domain-specific settings, where APIs may be idiosyncratic, under-documented, or tailored to private workflows.…
Learning When to Sample: Confidence-Aware Selective Sampling for Efficient Chain-of-Thought Reasoning
Juming Xiong, Kevin Guo, Congning Ni +7
Large language models (LLMs) can achieve strong reasoning performance through chain-of-thought (CoT) reasoning, yet they often generate unnecessarily long reasoning paths that incu…
CoRA: Confidence-Rationale Alignment for Reliable Chain-of-Thought Reasoning
Juming Xiong, Weixin Liu, Kevin Guo +9
Chain-of-thought (CoT) reasoning can improve LLM performance, but high answer confidence may be misleading when the accompanying CoT rationale is plausible yet incomplete or poorly…
When Evidence is Sparse: Weakly Supervised Early Failure Alerting in Dialogs and LLM-Agent Trajectories
Avinash Baidya, Xinran Liang, Ruocheng Guo +2
Early failure alerting requires deciding, while a dialog or agent trajectory is still unfolding, whether to flag it as likely to fail. This is challenging because supervision is ty…
It's Not Always Sycophancy: Measuring LLM Conformity as a Function of Epistemic Uncertainty
Kevin H. Guo, Chao Yan, Avinash Baidya +5
Large language models (LLMs) are known to abandon their initial stance to conform to user pushback. While prior research largely attributes this behavior to sycophancy learned duri…
Stop Listening to Me! How Multi-turn Conversations Can Degrade LLM Reliability
Kevin H. Guo, Chao Yan, Avinash Baidya +5
Large language models (LLMs) excel on static benchmarks, but their performance across multi-turn conversations, which better reflect real-world usage, remains understudied. Address…