6 papers
Same Patient, Different Order: Action-Level Reliability of Clinical LLM Agents Under Repeated Runs
Rohith Reddy Bellibatlu, Manpreet Singh, Zhoutian Han +1
A clinical agent benchmark can report the same verdict on identical inputs while the agent files a materially different order on each run. Such agents order tests, request medicati…
Instability Floors: Separating Bias from Noise in Fairness Audits of Clinical LLM Agents with FairMedAgent
Rohith Reddy Bellibatlu, Manpreet Singh, Deepak Parashar +1
Counterfactual fairness audits of clinical language-model agents report a flip rate: how often an action changes when only the patient's demographic descriptor changes. Part of tha…
Cost-Sensitive Conformal Prediction and Human-in-the-Loop Abstention for Imbalanced High-Stakes Decision Support: A Multi-Domain Benchmark
Manpreet Singh, Akshatha Srikantha, Shyamal Lakhanpal
High-stakes decision systems in credit scoring, fraud detection, healthcare, and industrial safety require reliable uncertainty quantification under severe class imbalance and asym…
TriEval: A Resource-Efficient Pipeline for LLM Bias, Toxicity, and Truthfulness Assessment
Akshatha Srikantha, Manpreet Singh, Yash Jajoo +1
LLMs have evolved from basic chatbots to the backbone of the AI ecosystem, now widely used in healthcare, schools, and government services. The domain-wide adoption of LLMs necessi…
AgentFairBench: Do LLM Agents Discriminate When They Act?
Triveni Morla, Rohith Reddy Bellibatlu, Manpreet Singh +1
Large language model (LLM) agents increasingly take actions (screening applicants, recommending credit, triaging patients), yet fairness for LLMs is still measured by grading answe…
RISED: A Pre-Deployment Evaluation Framework for High-Stakes AI Decision-Support Systems, with Application to Healthcare
Rohith Reddy Bellibatlu, Manpreet Singh, Yash Jajoo +2
Clinical decision-support systems are expert systems whose recommendations clinicians act on directly, yet they are usually cleared on one aggregate accuracy number from a held-out…