11 papers
Fantastic Adaptive Taxonomies and How to Use Them
Mert Cemri, Andrei Cojocaru, Melissa Pan +9
An agent system's execution traces record how it fails, and procedures that improve such a system without changing model weights (trajectory selection, prompt and workflow optimiza…
Measuring Agents in Production
Melissa Z. Pan, Negar Arabzadeh, Riccardo Cogo +22
LLM-based agents already operate in production across many industries, yet we lack an understanding of what technical methods make deployments successful. We present the first syst…
BenchEvolver: Frontier Task Synthesis via Solution-Centric Evolution
Yangzhen Wu, Aaron J. Li, Wenjie Ma +10
The rapid progress of frontier large language models has led to widespread benchmark saturation, limiting the ability of existing datasets to differentiate model capabilities or pr…
The Time is Here for Just-in-Time Systems: Challenges and Opportunities
Shu Liu, Alexander Krentsel, Shubham Agarwal +8
Core systems like key-value stores have historically taken years to build, and are designed to be general so as to amortize cost across deployments, paying a significant performanc…
Inductive Deductive Synthesis: Enabling AI to Generate Formally Verified Systems
Shubham Agarwal, Alexander Krentsel, Shu Liu +10
AI agents increasingly excel at generating, testing, and refining code. However, they fall short on tasks requiring formal guarantees of full coverage that testing alone cannot pro…
AdaEvolve: Adaptive LLM Driven Zeroth-Order Optimization
Mert Cemri, Shubham Agrawal, Akshat Gupta +9
The paradigm of automated program generation is shifting from one-shot generation to inference-time search, where Large Language Models (LLMs) function as semantic mutation operato…