2 citations · 2 across the 9 of their papers we have counts for
9 papers
DRACO: Fine-Grained Credit Assignment with Dynamic Rubrics for Long-Horizon Agent Training
Shubham Gandhi, Saurabh Goyal, Kiran Kate +1
Reinforcement Learning from Verifiable Rewards works well when a task has a programmatic checker, but most long-horizon agent domains have none. We work in the outcome-blind settin…
Agent Lifecycle Toolkit (ALTK): Reusable Middleware Components for Robust AI Agents
Zidane Wright, Jason Tsay, Anupama Murthi +8
As AI agents move from demos into enterprise deployments, their failure modes become consequential: a misinterpreted tool argument can corrupt production data, a silent reasoning e…
When Agents go Astray: Course-Correcting SWE Agents with PRMs
Shubham Gandhi, Jason Tsay, Jatin Ganhotra +2
Large Language Model (LLM) agents are increasingly deployed for complex, multi-step software engineering (SWE) tasks. However, their trajectories often contain costly inefficiencie…
Repairing Tool Calls Using Post-tool Execution Reflection and RAG
Jason Tsay, Zidane Wright, Gaodan Fang +3
Agentic systems interact with external systems by calling tools such as Python functions, REST API endpoints, or command line tools such as kubectl in Kubernetes. These tool calls…
How Good Are LLMs at Processing Tool Outputs?
Kiran Kate, Yara Rizk, Poulami Ghosh +4
Most realistic task automation problems require large language models (LLMs) to call tools, which often return complex JSON responses. These responses must be further processed to…
ToolRM: Outcome Reward Models for Tool-Calling Large Language Models
Mayank Agarwal, Ibrahim Abdelaziz, Kinjal Basu +4
As large language models (LLMs) increasingly interact with external tools, reward modeling for tool use has emerged as a critical yet underexplored area of research. Existing rewar…
LongFuncEval: Measuring the effectiveness of long context models for function calling
Kiran Kate, Tejaswini Pedapati, Kinjal Basu +5
Multiple recent studies have documented large language models' (LLMs) performance on calling external tools/functions. Others focused on LLMs' abilities to handle longer context le…
Towards LLMs Robustness to Changes in Prompt Format Styles
Lilian Ngweta, Kiran Kate, Jason Tsay +1
Large language models (LLMs) have gained popularity in recent years for their utility in various applications. However, they are sensitive to non-semantic changes in prompt formats…
NESTFUL: A Benchmark for Evaluating LLMs on Nested Sequences of API Calls
Kinjal Basu, Ibrahim Abdelaziz, Kiran Kate +10
The resurgence of autonomous agents built using large language models (LLMs) to solve complex real-world tasks has brought increased focus on LLMs' fundamental ability of tool or f…