5 papers
The Audit Decides the Verdict: Instrument Effects Rival Demographic Bias in LLM Decision Audits
Siddharth Vohra, Manikandan Ravikiran
Whether a language model looks demographically biased can depend on how the audit asks its question. A charitable-aid benchmark reports that the same models favor minority applican…
When Models Defer to Wrong Answers: A Robustness Audit of Source-Attributed Cues in Multiple-Choice QA
Manikandan Ravikiran, Siddharth Vohra
Language models often receive a question together with a claim about what another source answered. We audit whether such claims destabilize answers in multiple-choice question answ…
Does the Selected Object Reach the Reader? Auditing Identity Handoffs in Grounded Language-Model Pipelines
Siddharth Vohra, Runmin Jiang, Xiaomo Li +1
Grounded language-model pipelines can be divided into three stages: selecting an object, retrieving passages for it, and using that evidence to answer. If the selected object must…
Hearsay: Vision-Language Medical Diagnoses Without an Image
Siddharth Vohra
When asked to describe a medical image that was never attached, frontier vision-language models do not abstain: they confabulate a diagnosis. We show that this confabulation is not…
Investigating the Effect of Intraclass Variability in Temporal Ensembling
Siddharth Vohra, Manikandan Ravikiran
Temporal Ensembling is a semi-supervised approach that allows training deep neural network models with a small number of labeled images. In this paper, we present our preliminary s…