6 papers
Do Agent Benchmarks Do What They Say? An Executable-Contract Audit of Tool-Using Agent Environments
Rohith Reddy Bellibatlu, Zichong Wang, Wenbin Zhang
Tool-using agents are entering settings where a wrong action carries real cost, and the benchmarks certifying them grade what each simulated tool call reports having done, assuming…
Same Patient, Different Order: Action-Level Reliability of Clinical LLM Agents Under Repeated Runs
Rohith Reddy Bellibatlu, Manpreet Singh, Zhoutian Han +1
A clinical agent benchmark can report the same verdict on identical inputs while the agent files a materially different order on each run. Such agents order tests, request medicati…
Instability Floors: Separating Bias from Noise in Fairness Audits of Clinical LLM Agents with FairMedAgent
Rohith Reddy Bellibatlu, Manpreet Singh, Deepak Parashar +1
Counterfactual fairness audits of clinical language-model agents report a flip rate: how often an action changes when only the patient's demographic descriptor changes. Part of tha…
ExecuGraph: A Multi-Agent, Execution-Grounded Framework for Reliable Backend Code Synthesis with Large Language Models
Sai Deekshith Lekkala, Jothi Prabha Appadurai, Rohith Reddy Bellibatlu +1
Large Language Models generate plausible backend code, but a single-pass paradigm provides no guarantee of correctness or runtime reliability. We present ExecuGraph, a multi-agent…
RISED: A Pre-Deployment Evaluation Framework for High-Stakes AI Decision-Support Systems, with Application to Healthcare
Rohith Reddy Bellibatlu, Manpreet Singh, Yash Jajoo +2
Clinical decision-support systems are expert systems whose recommendations clinicians act on directly, yet they are usually cleared on one aggregate accuracy number from a held-out…
JudgeSense: A Benchmark for Prompt Sensitivity in LLM-as-a-Judge Systems
Rohith Reddy Bellibatlu, Edward Raff, Wenbin Zhang
Large language models are widely used to judge the output of other language models, yet whether a judge returns the same verdict when the same request is worded differently remains…