12 papers
Humans are Missing from AI Coding Agent Research
Zora Z. Wang, John Yang, Kilian Lieret +10
Recent progress in AI coding agent research has led to rapid improvements in agents' ability to autonomously perform complex software engineering tasks, from editing large codebase…
BuilderBench: The Building Blocks of Intelligent Agents
Raj Ghugare, Roger Creus Castanyer, Catherine Ji +4
Today's AI models learn primarily through mimicry and refining, so it is not surprising that they struggle to solve problems beyond the limits set by existing data. To solve novel…
Retaining by Doing: The Role of On-Policy Data in Mitigating Forgetting
Howard Chen, Noam Razin, Karthik Narasimhan +1
Adapting language models (LMs) to new tasks via post-training carries the risk of degrading existing capabilities -- a phenomenon classically known as catastrophic forgetting. In t…
-Rec: A Verifiable Benchmark for Agentic Recommender Systems
Bharath Sivaram Narasimhan, Karthik R Narasimhan
As recommender systems transition toward agentic, multi-turn conversational interfaces, evaluation paradigms have struggled to keep pace. Current benchmarks often rely on "LLM-as-a…
CEO-Bench: Can Agents Play the Long Game?
Haozhe Chen, Karthik Narasimhan, Zhuang Liu
Language model agents are becoming proficient executors at isolated, short-horizon tasks such as software engineering and customer service. Yet real-world challenges require a comb…
Context-Aware RL for Agentic and Multimodal LLMs
Peiyang Xu, Bangzheng Li, Sijia Liu +4
Large language models (LLMs) often fail when answering requires identifying a small but decisive piece of evidence within a long or complex context, such as a single line in a tool…