52 citations · 87 across the 14 of their papers we have counts for
11 papers · 1 filter
Cross-Benchmark Transfer from RL on Agentic Coding Tasks
Sushant Mehta, Logan Ritchie, Edwin Chen
Coding agents often fail in the last mile: they build most of a feature but drop a requirement, test only the cases their implementation already handles, break behavior that was su…
Chartography: A Benchmark for Professional Chart Understanding
Suhaas Garre, Chris Mutty, Sushant Mehta +1
Professionals across medicine, engineering, finance, manufacturing, and the sciences often make consequential decisions from charts. Existing chart benchmarks do not sufficiently m…
Post-Training on Office Work Improves Software Engineering: A Behavioral Account of Cross-Domain Transfer
Logan Ritchie, Sushant Mehta, Liudas Panavas +1
Long-horizon tasks require agents to maintain coherent state and goals across nested and branching work. We call this capability goal-directed execution (GDE): the repeated applica…
HANDBOOK.md: A Benchmark for Long-Context Agentic Instruction Following
Liudas Panavas, Sebastian Minus, Bradley Monton +4
Language-model agents are increasingly deployed under standing instructions: a system prompt, a policy file, or a skills document is placed in context, and the agent is trusted to…
Cross-Benchmark Generalization in Long-Horizon Agents
Sushant Mehta, Logan Ritchie, Liudas Panavas +1
For reinforcement learning (RL) in self-contained environments, a policy can get rewards by exploiting environment-specific regularities (tool schemas, grader parsing, task templat…
GDP.pdf: Benchmarking Grounded Multimodal Reasoning over Professional PDF Documents
Suhaas Garre, Emily Ritchie, Sushant Mehta +1
A large share of day-to-day work in professional domains happens inside PDF files: benefits packets, leases, datasheets, clinical guidelines, construction plans. Benchmarks for doc…
ComplexConstraints and Beyond: Expert Rubrics for RLVR
Sushant Mehta, Liudas Panavas, Suhaas Garre +1
Evaluation protocols can lag behind LLM capabilities. Programmatically verified benchmarks cover narrow surface constraints, whereas real-world instruction following and agentic wo…
Riemann-Bench: A Benchmark for Moonshot Mathematics
Suhaas Garre, Erik Knutsen, Sushant Mehta +1
Recent AI systems have achieved gold-medal-level performance on the International Mathematical Olympiad, demonstrating remarkable proficiency at competition-style problem solving.…
EnterpriseBench Corecraft: Training Generalizable Agents on High-Fidelity RL Environments
Sushant Mehta, Logan Ritchie, Suhaas Garre +3
We show that training AI agents on high-fidelity reinforcement learning environments produces capabilities that generalize beyond the training distribution. We introduce CoreCraft,…
Machine Learning as a Tool (MLAT): A Framework for Integrating Statistical ML Models as Callable Tools within LLM Agent Workflows
Edwin Chen, Zulekha Bibi
We introduce Machine Learning as a Tool (MLAT), a design pattern in which pre-trained statistical machine learning models are exposed as callable tools within large language model…
The Hierarchy of Agentic Capabilities: Evaluating Frontier Models on Realistic RL Environments
Logan Ritchie, Sushant Mehta, Nick Heiner +2
The advancement of large language model (LLM) based agents has shifted AI evaluation from single-turn response assessment to multi-step task completion in interactive environments.…