2 citations · 2 across the 8 of their papers we have counts for
6 papers · 1 filter
When Agents go Astray: Course-Correcting SWE Agents with PRMs
Shubham Gandhi, Jason Tsay, Jatin Ganhotra +2
Large Language Model (LLM) agents are increasingly deployed for complex, multi-step software engineering (SWE) tasks. However, their trajectories often contain costly inefficiencie…
Repairing Tool Calls Using Post-tool Execution Reflection and RAG
Jason Tsay, Zidane Wright, Gaodan Fang +3
Agentic systems interact with external systems by calling tools such as Python functions, REST API endpoints, or command line tools such as kubectl in Kubernetes. These tool calls…
How Good Are LLMs at Processing Tool Outputs?
Kiran Kate, Yara Rizk, Poulami Ghosh +4
Most realistic task automation problems require large language models (LLMs) to call tools, which often return complex JSON responses. These responses must be further processed to…
Live API-Bench: 2500+ Live APIs for Testing Multi-Step Tool Calling
Benjamin Elder, Anupama Murthi, Jungkoo Kang +4
Large language models (LLMs) increasingly rely on external tools and APIs to execute complex tasks specified in natural language. Evaluating such tool calling capabilities in reali…
LongFuncEval: Measuring the effectiveness of long context models for function calling
Kiran Kate, Tejaswini Pedapati, Kinjal Basu +5
Multiple recent studies have documented large language models' (LLMs) performance on calling external tools/functions. Others focused on LLMs' abilities to handle longer context le…
Towards LLMs Robustness to Changes in Prompt Format Styles
Lilian Ngweta, Kiran Kate, Jason Tsay +1
Large language models (LLMs) have gained popularity in recent years for their utility in various applications. However, they are sensitive to non-semantic changes in prompt formats…