14 papers
SEABED: SouthEast Asian Benchmark for Evaluating Audio Reasoning
Harshit Rajgarhia, Asif Shaik, Rachuri Lokesh +2
Modern audio-language models are no longer judged only on what words they can transcribe, but on whether they can reason over what they hear: recovering meaning that lives in tone…
Voice or Stereotype? Disentangling Acoustic and Content-Based Gender in Speech-to-Speech Models
Xiaoqun Liu, Tanu Mitra, Harshit Rajgarhia +1
Speech-to-speech (S2S) models now run inside dubbing, translation, and voice agents. Unlike text models, they hear the speaker's voice, which carries the speaker's gender. A faithf…
RL-ADA: A World-Feedback Framework for Adversarially Robust Enterprise Dialogue Agents
Ram Narayanan, Harshit Rajgarhia, Abhishek Mukherji
Deploying task-oriented dialogue agents in enterprise customer support faces a persistent annotation bottleneck: robust training requires labelled interaction data at scale, yet en…
DuplexWorld: Can voice agents help you get through the day?
Aryan Vijay Bhosale, Harshit Rajgarhia, Akhil Pothanapalli +3
Speech-to-speech (S2S) voice agents are increasingly being incorporated into enterprise for customer care and as daily companions for consumers owing to the ease of the conversatio…
TORUS: A Unified Closed-Loop Benchmark for Unified Audio Models
Aryan Vijay Bhosale, Harshit Rajgarhia, Abhishek Mukherji +1
Unified audio models capable of audio understanding, audio generation and, increasingly, audio editing are proliferating rapidly. However, each capability is evaluated in isolation…
CaReCoS: A Spectrogram based Visual Benchmark for Cardiac, Respiratory and Cough Sounds
Harshit Rajgarhia, Shuubham Ojha, Akhil Pothanapalli +4
Medical acoustic signals such as respiratory sounds, cardiac auscultations, and cough audio carry rich diagnostic information, yet no existing benchmark evaluates multimodal reason…
World Feedback for Clinical Agents: Diagnosing RL in FHIR Environments
Ananya Mantravadi, Harshit Rajgarhia, Prasanna Desikan +1
Clinical protocol-execution tasks -- checking a lab value, applying a threshold, placing a correctly structured FHIR order -- are natural candidates for RL from world feedback: onc…
Beyond Next-Token Prediction: An RLVR Proof of Concept for Tool-Use Agents on Atlassian Workflows
Karthikeya Aditya Vissa, Sankalp Mane, Ananya Mantravadi +2
Large language models are trained to predict the next token, not to act inside a specific API. In niche enterprise SaaS workflows -- where success means hitting the right endpoint…
MedMosaic: A Challenging Large Scale Benchmark of Diverse Medical Audio
Harshit Rajgarhia, Shuubham Ojha, Asif Shaik +4
Medical audio data is difficult to collect due to privacy regulations and high annotation costs arising from domain expertise. Thus, existing benchmarks tend to underrepresent comp…
Measuring What Matters: Benchmarking Generative, Multimodal, and Agentic AI in Healthcare
Prasanna Desikan, Harshit Rajgarhia, Shivali Dalmia +1
AI models are increasingly deployed in live clinical environments where they must perform reliably across complex, high-stakes workflows that standard training and validation datas…
Scalable multilingual PII annotation for responsible AI in LLMs
Bharti Meena, Joanna Skubisz, Harshit Rajgarhia +5
As Large Language Models (LLMs) gain wider adoption, ensuring their reliable handling of Personally Identifiable Information (PII) across diverse regulatory contexts has become ess…
An Evaluation Study of Hybrid Methods for Multilingual PII Detection
Harshit Rajgarhia, Suryam Gupta, Asif Shaik +4
The detection of Personally Identifiable Information (PII) is critical for privacy compliance but remains challenging in low-resource languages due to linguistic diversity and limi…
GAZE:Governance-Aware pre-annotation for Zero-shot World Model Environments
Leela Krishna, Mengyang Zhao, Saicharithreddy Pasula +2
Training robust world models requires large-scale, precisely labeled multimodal datasets, a process historically bottlenecked by slow and expensive manual annotation. We present a…
Human + AI for Accelerating Ad Localization Evaluation
Harshit Rajgarhia, Shivali Dalmia, Mengyang Zhao +2
Adapting advertisements for multilingual audiences requires more than simple text translation; it demands preservation of visual consistency, spatial alignment, and stylistic integ…