2 citations · 5 across the 15 of their papers we have counts for
8 papers · 1 filter
Last Translation Benchmark
Vilém Zouhar, Niyati Bafna, Mukund Choudhary +263
For scientific progress, we need benchmarks that test the limits of state-of-the-art models, and evaluation methods that inform us about failure cases. As models get stronger, stan…
SN-WER: Script-Normalized WER for Multi-Script Indic ASR Evaluation
Priyaranjan Pattnayak
Word Error Rate (WER) is the dominant metric for automatic speech recognition (ASR), but it can overestimate errors when references and hypotheses encode the same words in differen…
IndicSafe: A Benchmark for Evaluating Multilingual LLM Safety in South Asia
Priyaranjan Pattnayak, Sanchari Chowdhuri
As large language models (LLMs) are deployed in multilingual settings, their safety behavior in culturally diverse, low-resource languages remains poorly understood. We present the…
SweEval: Do LLMs Really Swear? A Safety Benchmark for Testing Limits for Enterprise Use
Hitesh Laxmichand Patel, Amit Agarwal, Arion Das +6
Enterprise customers are increasingly adopting Large Language Models (LLMs) for critical communication tasks, such as drafting emails, crafting sales pitches, and composing casual…
Tokenization Matters: Improving Zero-Shot NER for Indic Languages
Priyaranjan Pattnayak, Hitesh Laxmichand Patel, Amit Agarwal
Tokenization is a critical component of Natural Language Processing (NLP), especially for low resource languages, where subword segmentation influences vocabulary structure and dow…
Clinical QA 2.0: Multi-Task Learning for Answer Extraction and Categorization
Priyaranjan Pattnayak, Hitesh Laxmichand Patel, Amit Agarwal +3
Clinical Question Answering (CQA) plays a crucial role in medical decision-making, enabling physicians to extract relevant information from Electronic Medical Records (EMRs). While…
LLM for Barcodes: Generating Diverse Synthetic Data for Identity Documents
Hitesh Laxmichand Patel, Amit Agarwal, Bhargava Kumar +2
Accurate barcode detection and decoding in Identity documents is crucial for applications like security, healthcare, and education, where reliable data extraction and verification…
Enhancing Document AI Data Generation Through Graph-Based Synthetic Layouts
Amit Agarwal, Hitesh Patel, Priyaranjan Pattnayak +3
The development of robust Document AI models has been constrained by limited access to high-quality, labeled datasets, primarily due to data privacy concerns, scarcity, and the hig…