6 papers
Same Entity, Different Script: Diagnosing Cross-Script Entity Matching Failures in Language Models
Athira Gopal, Patil Prathamesh Pravin, Ankush Kadu +2
Multilingual language models can give different identity judgments when the same people's names are written in different scripts. We study this effect using 601 audited identity pa…
Active Causal Discovery Benchmark: Evaluating LLM Agents Under Budgeted Interventions
Sagar Deb, Devam Shah, Ashwanth Krishnan
We introduce the Active Causal Discovery Benchmark (ACDB), an SCM-grounded environment for evaluating whether LLM agents recover causal graph structure from observations and budget…
STOCKTAKE: Measuring the Gap Between Perception and Action in LLM Agents with a Fair Oracle
Sagar Deb, Ashwanth Krishnan
LLM agents are increasingly evaluated on multi-week decision tasks in which the state that drives cost is never directly observed. On such tasks the final cost cannot say why an ag…
How Far Can Root Cause Analysis Go on Real-World Telemetry Data?
Athira Gopal, Ashwanth Krishnan
Root cause analysis (RCA) of microservice failures requires reasoning over telemetry far larger than a model's context, so a language agent reasons over extracted evidence. We star…
ReTreVal: Reasoning Tree with Validation and Cross-Problem Memory for Large Language Models
Abhishek HS, Pavan C Shekar, Arpit Jain +1
Every existing inference-time reasoning framework discards all failure context at problem boundaries, leaving a model solving problem 500 no wiser than it was on problem 1. We pres…
Adaptive Minds: Empowering Agents with LoRA-as-Tools
Pavan C Shekar, Aswanth Krishnan
We investigate a framework in which LoRA adapters are treated as callable tools that a base language model can dynamically select and invoke. We hypothesize that, when adapters are…