12 papers
A Trust Layer for Agent Evaluation
Mohammadreza Sediqin, Shivali Dalmia, Srinivasa Karthikeya Reddy Kovvuri +1
Deterministic benchmark scores show that an agent received credit, but not whether that credit was earned, reported honestly, or would hold on a second run. We introduce a Trust La…
Same Output, Different Gold: Measuring How Reference Choice Moves a Multilingual Benchmark Score
Parth Kulshreshtha, Shivali Dalmia, Abhishek Mukherji
A benchmark score compares a system output against a reference, and methodological attention falls almost entirely on the first term. We measure the second. The retained annotation…
SAGE: Governed Artifact Generation from Enterprise Guidelines
Mohammadreza Sediqin, Shivali Dalmia, Sumukha Thoppanahalli +2
Enterprise guideline documents mix narrative text, complex tables, and embedded images, and converting them into structured work artifacts still takes two to three days of manual e…
GVD: Governed Versioning and Deduplication for Document Repositories
Mohammadreza Sediqin, Shivali Dalmia, Sumukha Thoppanahalli +1
Document repositories evolve continuously. Guidelines and policies are revised, superseded, and re-uploaded, so the same content recurs in different wording and newer versions refi…
GUIDE: Governed Unified Intelligence for Document-to-Artifact Generation in Enterprise Settings
Shivali Dalmia, Sumukha Thoppanahalli, Mohammadreza Sediqin +1
Enterprise guideline documents are heterogeneous and multimodal, combining narrative text, complex tables, and embedded images. Existing LLM and VLM systems face hallucinated conte…
Measuring What Matters: Benchmarking Generative, Multimodal, and Agentic AI in Healthcare
Prasanna Desikan, Harshit Rajgarhia, Shivali Dalmia +1
AI models are increasingly deployed in live clinical environments where they must perform reliably across complex, high-stakes workflows that standard training and validation datas…
Systematic Evaluation of Large Language Models for Post-Discharge Clinical Action Extraction
Shivali Dalmia, Ananya Mantravadi, Prasanna Desikan
The work in this paper evaluates zero-shot and few-shot large language models (LLMs) for safety-critical clinical action extraction using the CLIP discharge-note dataset, with part…
CareTransition-Audit: A Benchmark to Audit Discharge Summaries for Efficient Care Transitions
Akshat Dasula, Prasanna Desikan, Jaideep Srivastava +2
Incomplete or inconsistent discharge documentation drives care fragmentation and avoidable readmissions. Despite its critical role in patient safety, auditing discharge summaries r…
ART: Action-based Reasoning Task Benchmarking for Medical AI Agents
Ananya Mantravadi, Shivali Dalmia, Abhishek Mukherji
Reliable clinical decision support requires medical AI agents capable of safe, multi-step reasoning over structured electronic health records (EHRs). While large language models (L…
LegalWiz: A Multi-Agent Generation Framework for Contradiction Detection in Legal Documents
Ananya Mantravadi, Shivali Dalmia, Olga Pospelova +3
Retrieval-Augmented Generation (RAG) integrates large language models (LLMs) with external sources, but unresolved contradictions in retrieved evidence often lead to hallucinations…
Scalable multilingual PII annotation for responsible AI in LLMs
Bharti Meena, Joanna Skubisz, Harshit Rajgarhia +5
As Large Language Models (LLMs) gain wider adoption, ensuring their reliable handling of Personally Identifiable Information (PII) across diverse regulatory contexts has become ess…
Human + AI for Accelerating Ad Localization Evaluation
Harshit Rajgarhia, Shivali Dalmia, Mengyang Zhao +2
Adapting advertisements for multilingual audiences requires more than simple text translation; it demands preservation of visual consistency, spatial alignment, and stylistic integ…