7 papers · 1 filter
CRAB-Bench: Evaluating LLM Agents under Complex Task Dependencies and Human-aligned User Simulation
Danqing Wang, Akshay Sivaraman, Lei Li
Evaluating LLM agents in realistic service scenarios requires complex task dependencies, imperfect user behavior, and an evaluation that accommodates multiple valid solutions. We i…
Learning to Interrupt in Language-based Multi-agent Communication
Danqing Wang, Da Yin, Ruta Desai +3
When a colleague starts explaining something you already understand, you interrupt them. This simple act, a listener taking control of the conversation, is natural in human communi…
Strategic Planning and Rationalizing on Trees Make LLMs Better Debaters
Danqing Wang, Zhuorui Ye, Xinran Zhao +2
Winning competitive debates requires sophisticated reasoning and argument skills. There are unique challenges in the competitive debate: (1) The time constraints force debaters to…
RARE: Retrieval-Aware Robustness Evaluation for Retrieval-Augmented Generation Systems
Yixiao Zeng, Tianyu Cao, Danqing Wang +5
Retrieval-Augmented Generation (RAG) enhances recency and factuality in answers. However, existing evaluations rarely test how well these systems cope with real-world noise, confli…
TypedThinker: Diversify Large Language Model Reasoning with Typed Thinking
Danqing Wang, Jianxin Ma, Fei Fang +1
Large Language Models (LLMs) have demonstrated strong reasoning capabilities in solving complex problems. However, current approaches primarily enhance reasoning through the elabor…
Scaling LLM Inference with Optimized Sample Compute Allocation
Kexun Zhang, Shang Zhou, Danqing Wang +2
Sampling is a basic operation in many inference-time algorithms of large language models (LLMs). To scale up inference efficiently with a limited compute, it is crucial to find an…