8 papers
UXBench: Measuring the Actionability of LLM-Generated UX Critiques
Wenjie Wang, Yue Huang, Zipeng Ling +11
Large language models (LLMs) are increasingly deployed as UX judges that inspect interfaces, diagnose usability problems, and propose repairs. Yet no controlled benchmark measures…
When Simulation Lies: A Sim-to-Real Benchmark and Domain-Randomized RL Recipe for Tool-Use Agents
Xiaolin Zhou, Aojie Yuan, Zheng Luo +12
Tool-use language agents are evaluated on benchmarks that assume clean inputs, unambiguous tool registries, and reliable APIs. Real deployments violate all these assumptions: user…
NARRA-Gym for Evaluating Interactive Narrative Agents
Yue Huang, Yuchen Ma, Jiayi Ye +14
Interactive narrative tasks require LLMs to sustain a coherent, evolving story while adapting to a user over multiple turns. However, suitable benchmarks for this setting are limit…
Omni-SimpleMem: Autoresearch-Guided Discovery of Lifelong Multimodal Agent Memory
Jiaqi Liu, Zipeng Ling, Shi Qiu +9
AI agents increasingly operate over extended time horizons, yet their ability to retain, organize, and recall multimodal experiences remains a critical bottleneck. Building effecti…
Correct Prediction, Wrong Steps? Consensus Reasoning Knowledge Graph for Robust Chain-of-Thought Synthesis
Zipeng Ling, Shuliang Liu, Seonil Son +4
Large language models (LLMs) have become increasingly used for various tasks, often coupled with Chain-of-Thought (CoT) prompting to boost accuracy. Recent work has shown that high…
Quantifying LLM Biases Across Instruction Boundary in Mixed Question Forms
Zipeng Ling, Shuliang Liu, Yuehao Tang +8
Large Language Models (LLMs) annotated datasets are widely used nowadays, however, large-scale annotations often show biases in low-quality datasets. For example, Multiple-Choice Q…
LLM Abstention Can Be a Prompt Artifact, in Addition to Genuine Uncertainty
Zipeng Ling, Shuliang Liu, Yuehao Tang +8
Large Language Models (LLMs) are increasingly trained to abstain from answering questions they are unsure about. However, this ability is often misapplied: in real-world applicatio…
Dissecting Logical Reasoning in LLMs: A Fine-Grained Evaluation and Supervision Study
Yujun Zhou, Jiayi Ye, Zipeng Ling +8
Logical reasoning is a core capability for large language models (LLMs), yet existing benchmarks that rely solely on final-answer accuracy fail to capture the quality of the reason…