12 papers · 1 filter
A Trust Layer for Agent Evaluation
Mohammadreza Sediqin, Shivali Dalmia, Srinivasa Karthikeya Reddy Kovvuri +1
Deterministic benchmark scores show that an agent received credit, but not whether that credit was earned, reported honestly, or would hold on a second run. We introduce a Trust La…
CuBEs: Culturally-Situated Behavioral Evaluations and the Limitations of Culture-Blind LLM Judges
Hoda Ayad, Tanu Mitra, Abhishek Mukherji
Evaluating the occurrence and triggers of large language model (LLM) behaviors - such as sycophancy, self-preference, or over-confidence - is critical for predicting real-world mod…
SAGE: Governed Artifact Generation from Enterprise Guidelines
Mohammadreza Sediqin, Shivali Dalmia, Sumukha Thoppanahalli +2
Enterprise guideline documents mix narrative text, complex tables, and embedded images, and converting them into structured work artifacts still takes two to three days of manual e…
GVD: Governed Versioning and Deduplication for Document Repositories
Mohammadreza Sediqin, Shivali Dalmia, Sumukha Thoppanahalli +1
Document repositories evolve continuously. Guidelines and policies are revised, superseded, and re-uploaded, so the same content recurs in different wording and newer versions refi…
GUIDE: Governed Unified Intelligence for Document-to-Artifact Generation in Enterprise Settings
Shivali Dalmia, Sumukha Thoppanahalli, Mohammadreza Sediqin +1
Enterprise guideline documents are heterogeneous and multimodal, combining narrative text, complex tables, and embedded images. Existing LLM and VLM systems face hallucinated conte…
World Feedback for Clinical Agents: Diagnosing RL in FHIR Environments
Ananya Mantravadi, Harshit Rajgarhia, Prasanna Desikan +1
Clinical protocol-execution tasks -- checking a lab value, applying a threshold, placing a correctly structured FHIR order -- are natural candidates for RL from world feedback: onc…
Beyond Next-Token Prediction: An RLVR Proof of Concept for Tool-Use Agents on Atlassian Workflows
Karthikeya Aditya Vissa, Sankalp Mane, Ananya Mantravadi +2
Large language models are trained to predict the next token, not to act inside a specific API. In niche enterprise SaaS workflows -- where success means hitting the right endpoint…
CareTransition-Audit: A Benchmark to Audit Discharge Summaries for Efficient Care Transitions
Akshat Dasula, Prasanna Desikan, Jaideep Srivastava +2
Incomplete or inconsistent discharge documentation drives care fragmentation and avoidable readmissions. Despite its critical role in patient safety, auditing discharge summaries r…
ART: Action-based Reasoning Task Benchmarking for Medical AI Agents
Ananya Mantravadi, Shivali Dalmia, Abhishek Mukherji
Reliable clinical decision support requires medical AI agents capable of safe, multi-step reasoning over structured electronic health records (EHRs). While large language models (L…
LegalWiz: A Multi-Agent Generation Framework for Contradiction Detection in Legal Documents
Ananya Mantravadi, Shivali Dalmia, Olga Pospelova +3
Retrieval-Augmented Generation (RAG) integrates large language models (LLMs) with external sources, but unresolved contradictions in retrieved evidence often lead to hallucinations…
An Evaluation Study of Hybrid Methods for Multilingual PII Detection
Harshit Rajgarhia, Suryam Gupta, Asif Shaik +4
The detection of Personally Identifiable Information (PII) is critical for privacy compliance but remains challenging in low-resource languages due to linguistic diversity and limi…
Human + AI for Accelerating Ad Localization Evaluation
Harshit Rajgarhia, Shivali Dalmia, Mengyang Zhao +2
Adapting advertisements for multilingual audiences requires more than simple text translation; it demands preservation of visual consistency, spatial alignment, and stylistic integ…