7 papers · 1 filter
Can an Agent Learn to Predict Its Own Failure? Outcome-Grounded Confidence Calibration in LLM Agents
Ashutosh Hathidara
Controlling an acting LLM agent means deciding, at each step of a long-horizon task, whether to trust its next action, abstain, or hand off to a human. That decision needs a per-st…
Synthesis Through Simulation: Generating Coherent Enterprise Data via Scalable Agent-System Interaction
Yipeng Li, Ashutosh Hathidara, Jane Lo +3
Tool-calling agents have become central to enterprise AI, yet training and evaluating them at scale remains severely constrained due to business and legal restrictions on enterpris…
EDGEGEN: Improving Tool-Calling Agents Beyond Happy Paths with Synthetic Edge Case Generation
Harshavardhan Abichandani, Penny Chong, Jiyuan Shen +7
Tool-calling LLM agents are increasingly deployed in enterprise applications. However, effective evaluation and optimization require high-quality, diverse task datasets that are of…
TRACE: Business Rule-Grounded Reasoning Curriculum for Knowledge-Preserving Parametric Tool Retrieval in Enterprise LLMs
Sai Shruthi Sistla, Ashutosh Hathidara, Christopher Toukmaji +2
Parametric retrieval enables LLMs to retrieve tools implicitly by assigning each API a unique virtual token and training the model to generate it via constrained beam search. Tools…
ToolSense: A Diagnostic Framework for Auditing Parametric Tool Knowledge in LLMs
Ashutosh Hathidara, Sai Shruthi Sistla, Sebastian Schreiber +1
Large language models deployed as agents over large tool catalogs face a critical tool-retrieval bottleneck. As embedding-based retrieval approaches rely on compact encoders that m…
CoHyDE: Iterative Co-Training of LLM Rewriter & Dense Encoder for Tool Retrieval
Vaishali Senthil, Ashutosh Hathidara, Sebastian Schreiber
Tool retrieval over large API catalogs is a core bottleneck for LLM agents: user queries arrive in colloquial, often underspecified language, while the catalog uses technical API v…
MirrorBench: A Benchmark to Evaluate Conversational User-Proxy Agents for Human-Likeness
Ashutosh Hathidara, Julien Yu, Vaishali Senthil +2
Large language models (LLMs) are increasingly used as human simulators, both for evaluating conversational systems and for generating fine-tuning data. However, naive "act-as-a-use…