7 papers · 1 filter
Up and Down the Abstraction Ladder: Code-Based Skills for Language Agents
Bartłomiej Cupiał, Jens Tuyls, Maciej Wołczyk +5
Language agents struggle to act and learn in environments that require long sequences of low-level actions. Code-based abstractions can make these agents more productive by letting…
-Bench: An Environment for End-To-End, Realistic Agent Construction
Quan Shi, Keshav Dhandhania, Karthik Narasimhan +1
LLM agents are rapidly becoming production software, deployed to handle customer service, adjudicate disputes, and operate internal systems. Notably, the work of building them is i…
CEO-Bench: Can Agents Play the Long Game?
Haozhe Chen, Karthik Narasimhan, Zhuang Liu
Language model agents are becoming proficient executors at isolated, short-horizon tasks such as software engineering and customer service. Yet real-world challenges require a comb…
-Knowledge: Evaluating Conversational Agents over Unstructured Knowledge
Quan Shi, Alexandra Zytek, Pedram Razavi +2
Conversational agents are increasingly deployed in knowledge-intensive settings, where correct behavior depends on retrieving and applying domain-specific knowledge from large, pro…
BuilderBench: The Building Blocks of Intelligent Agents
Raj Ghugare, Roger Creus Castanyer, Catherine Ji +4
Today's AI models learn primarily through mimicry and refining, so it is not surprising that they struggle to solve problems beyond the limits set by existing data. To solve novel…
When Models Know More Than They Can Explain: Quantifying Knowledge Transfer in Human-AI Collaboration
Quan Shi, Carlos E. Jimenez, Shunyu Yao +3
Recent advancements in AI reasoning have driven substantial improvements across diverse tasks. A critical open question is whether these improvements also yields better knowledge t…
Contextual Experience Replay for Self-Improvement of Language Agents
Yitao Liu, Chenglei Si, Karthik Narasimhan +1
Large language model (LLM) agents have been applied to sequential decision-making tasks such as web navigation, but without any environment-specific experiences, they often fail in…