5 papers
SWE-Clar: Can Coding Agents Clarify Incomplete Client Requirements and Deliver Greenfield Software?
Jialei Li, Dingxian Wang, Xiaoyan Zhao +11
Greenfield software development in freelance settings requires coding agents to turn incomplete client requests into working software. This makes requirement clarification essentia…
Reinforcement Learning with Decomposed Subtasks
Mattie Terzolo, Mikolaj Sacha, Ayan Sinha +1
Group Relative Policy Optimization (GRPO) and related policy-gradient methods for training language model agents collapse an entire multi-turn rollout into a single scalar trajecto…
Evaluating Language Models in Realistic Conversational Contexts
Ilija Subasic, Andrew Rabinovich, Zhao Chen
As Large Language Models (LLMs) are increasingly deployed to serve open-ended, multi-turn interactions, evaluating conversational quality at human scale has become a central challe…
UpBench: A Dynamically Evolving Real-World Labor-Market Agentic Benchmark Framework Built for Human-Centric AI
Darvin Yi, Teng Liu, Mattie Terzolo +4
As large language model (LLM) agents increasingly undertake digital work, reliable frameworks are needed to evaluate their real-world competence, adaptability, and capacity for hum…
GraphMatch: Fusing Language and Graph Representations in a Dynamic Two-Sided Work Marketplace
Mikołaj Sacha, Hammad Jafri, Mattie Terzolo +2
Recommending matches in a text-rich, dynamic two-sided marketplace presents unique challenges due to evolving content and interaction graphs. We introduce GraphMatch, a new large-s…