19 papers
From Passive Delegates to Strategic Negotiators: Reinforcing Social Reasoning in Small Language Models with SocialRL
Wenyue Hua, Zachary Huang, Tyler Payne +3
AI agents increasingly act on their users' behalf, handling tasks such as scheduling meetings, comparing offers, and haggling over prices. These principal-driven tasks routinely pl…
AgentOpt v0.1 Technical Report: Client-Side Optimization for LLM-Based Agent
Wenyue Hua, Sripad Karne, Qian Xie +4
AI agents are increasingly deployed in real-world applications, including systems such as Manus, OpenClaw, and coding agents. Existing research has primarily focused on server-side…
Chimera: Latency- and Performance-Aware Multi-agent Serving for Heterogeneous LLMs
Kangqi Ni, Wenyue Hua, Xiaoxiang Shi +3
Multi-agent applications often execute complex tasks as multi-stage workflows, where each stage is an LLM call whose output becomes part of context for subsequent steps. Existing L…
Bridging Graph Structure and Knowledge-Guided Editing for Interpretable Temporal Knowledge Graph Reasoning
Shiqi Fan, Quanming Yao, Hongyi Nie +3
Temporal knowledge graph reasoning (TKGR) aims to predict future events by inferring missing entities with dynamic knowledge structures. Existing LLM-based reasoning methods priori…
Epistemic Context Learning: Building Trust the Right Way in LLM-Based Multi-Agent Systems
Ruiwen Zhou, Maojia Song, Xiaobao Wu +8
Individual agents in multi-agent (MA) systems often lack robustness, tending to blindly conform to misleading peers. We show this weakness stems from both sycophancy and inadequate…
Can Online GenAI Discussion Serve as Bellwether for Labor Market Shifts?
Shurui Cao, Wenyue Hua, William Yang Wang +2
The rapid advancement of Large Language Models (LLMs) has generated considerable speculation regarding their transformative potential for labor markets. However, existing approache…
Magentic Marketplace: An Open-Source Environment for Studying Agentic Markets
Gagan Bansal, Wenyue Hua, Zezhou Huang +21
As LLM agents advance, they are increasingly mediating economic decisions, ranging from product discovery to transactions, on behalf of users. Such applications promise benefits bu…
MAGPIE: A benchmark for Multi-AGent contextual PrIvacy Evaluation
Gurusha Juneja, Jayanth Naga Sai Pasupulati, Alon Albalak +2
A core challenge for autonomous LLM agents in collaborative settings is balancing robust privacy understanding and preservation alongside task efficacy. Existing privacy benchmarks…
Dynamic Speculative Agent Planning
Yilin Guan, Qingfeng Lan, Sun Fei +5
Despite their remarkable success in complex tasks propelling widespread adoption, large language-model-based agents still face critical deployment challenges due to prohibitive lat…
MAGPIE: A dataset for Multi-AGent contextual PrIvacy Evaluation
Gurusha Juneja, Alon Albalak, Wenyue Hua +1
The proliferation of LLM-based agents has led to increasing deployment of inter-agent collaboration for tasks like scheduling, negotiation, resource allocation etc. In such systems…
Semantic Scheduling for LLM Inference
Wenyue Hua, Dujian Ding, Yile Gu +4
Conventional operating system scheduling algorithms are largely content-ignorant, making decisions based on factors such as latency or fairness without considering the actual inten…
THOUGHTTERMINATOR: Benchmarking, Calibrating, and Mitigating Overthinking in Reasoning Models
Xiao Pu, Michael Saxon, Wenyue Hua +1
Reasoning models have demonstrated impressive performance on difficult tasks that traditional language models struggle at. However, many are plagued with the problem of overthinkin…
Towards a Design Guideline for RPA Evaluation: A Survey of Large Language Model-Based Role-Playing Agents
Chaoran Chen, Bingsheng Yao, Ruishi Zou +5
Role-Playing Agent (RPA) is an increasingly popular type of LLM Agent that simulates human-like behaviors in a variety of tasks. However, evaluating RPAs is challenging due to dive…
REALM: A Dataset of Real-World LLM Use Cases
Jingwen Cheng, Kshitish Ghate, Wenyue Hua +3
Large Language Models (LLMs), such as the GPT series, have driven significant industrial applications, leading to economic and societal transformations. However, a comprehensive un…
SOPBench: Evaluating Language Agents at Following Standard Operating Procedures and Constraints
Zekun Li, Shinda Huang, Jiangtian Wang +8
As language agents increasingly automate critical tasks, their ability to follow domain-specific standard operating procedures (SOPs), policies, and constraints when taking actions…
InductionBench: LLMs Fail in the Simplest Complexity Class
Wenyue Hua, Tyler Wong, Sun Fei +3
Large language models (LLMs) have shown remarkable improvements in reasoning and many existing benchmarks have been addressed by models such as o1 and o3 either fully or partially.…
Attention Mechanism for LLM-based Agents Dynamic Diffusion under Information Asymmetry
Yiwen Zhang, Yifu Wu, Wenyue Hua +2
Large language models have been used to simulate human society using multi-agent systems. Most current social simulation research emphasizes interactive behaviors in fixed environm…
RuleArena: A Benchmark for Rule-Guided Reasoning with LLMs in Real-World Scenarios
Ruiwen Zhou, Wenyue Hua, Liangming Pan +4
This paper introduces RuleArena, a novel and challenging benchmark designed to evaluate the ability of large language models (LLMs) to follow complex, real-world rules in reasoning…
Propagation and Pitfalls: Reasoning-based Assessment of Knowledge Editing through Counterfactual Tasks
Wenyue Hua, Jiang Guo, Mingwen Dong +3
Current approaches of knowledge editing struggle to effectively propagate updates to interconnected facts. In this work, we delve into the barriers that hinder the appropriate prop…