10 papers · 1 filter
Can an Agent Learn to Predict Its Own Failure? Outcome-Grounded Confidence Calibration in LLM Agents
Ashutosh Hathidara
Controlling an acting LLM agent means deciding, at each step of a long-horizon task, whether to trust its next action, abstain, or hand off to a human. That decision needs a per-st…
Synthesis Through Simulation: Generating Coherent Enterprise Data via Scalable Agent-System Interaction
Yipeng Li, Ashutosh Hathidara, Jane Lo +3
Tool-calling agents have become central to enterprise AI, yet training and evaluating them at scale remains severely constrained due to business and legal restrictions on enterpris…
EDGEGEN: Improving Tool-Calling Agents Beyond Happy Paths with Synthetic Edge Case Generation
Harshavardhan Abichandani, Penny Chong, Jiyuan Shen +7
Tool-calling LLM agents are increasingly deployed in enterprise applications. However, effective evaluation and optimization require high-quality, diverse task datasets that are of…
TRACE: Business Rule-Grounded Reasoning Curriculum for Knowledge-Preserving Parametric Tool Retrieval in Enterprise LLMs
Sai Shruthi Sistla, Ashutosh Hathidara, Christopher Toukmaji +2
Parametric retrieval enables LLMs to retrieve tools implicitly by assigning each API a unique virtual token and training the model to generate it via constrained beam search. Tools…
ToolSense: A Diagnostic Framework for Auditing Parametric Tool Knowledge in LLMs
Ashutosh Hathidara, Sai Shruthi Sistla, Sebastian Schreiber +1
Large language models deployed as agents over large tool catalogs face a critical tool-retrieval bottleneck. As embedding-based retrieval approaches rely on compact encoders that m…
CoHyDE: Iterative Co-Training of LLM Rewriter & Dense Encoder for Tool Retrieval
Vaishali Senthil, Ashutosh Hathidara, Sebastian Schreiber
Tool retrieval over large API catalogs is a core bottleneck for LLM agents: user queries arrive in colloquial, often underspecified language, while the catalog uses technical API v…
MirrorBench: A Benchmark to Evaluate Conversational User-Proxy Agents for Human-Likeness
Ashutosh Hathidara, Julien Yu, Vaishali Senthil +2
Large language models (LLMs) are increasingly used as human simulators, both for evaluating conversational systems and for generating fine-tuning data. However, naive "act-as-a-use…
Disambiguation-Centric Finetuning Makes Enterprise Tool-Calling LLMs More Realistic and Less Risky
Ashutosh Hathidara, Julien Yu, Sebastian Schreiber
Large language models (LLMs) are increasingly tasked with invoking enterprise APIs, yet they routinely falter when near-duplicate tools vie for the same user intent or when require…
Mining Tweets to Predict Future Bitcoin Price
Ashutosh Hathidara, Gaurav Atavale, Suyash Chaudhary
Bitcoin has increased investment interests in people during the last decade. We have seen an increase in the number of posts on social media platforms about cryptocurrency, especia…
Neuro-Symbolic Sudoku Solver
Ashutosh Hathidara, Lalit Pandey
Deep Neural Networks have achieved great success in some of the complex tasks that humans can do with ease. These include image recognition/classification, natural language process…