4 papers
Voice of India: A Large-Scale Benchmark for Real-World Speech Recognition in India
Kaushal Bhogale, Manas Dhir, Amritansh Walecha +11
Existing Indic ASR benchmarks often use scripted, clean speech and leaderboard driven evaluation that encourages dataset specific overfitting. In addition, strict single reference…
Benchmarking Speech Systems for Frontline Health Conversations: The DISPLACE-M Challenge
Dhanya E, Ankita Meena, Manas Nanivadekar +11
The DIarization and Speech Processing for LAnguage understanding in Conversational Environments - Medical (DISPLACE-M) challenge introduces a conversational AI benchmark for unders…
Investigating Distributions of Telecom Adapted Sentence Embeddings for Document Retrieval
Sujoy Roychowdhury, Sumit Soman, Ranjani Hosakere Gireesha +4
A plethora of sentence embedding models makes it challenging to choose one, especially for technical domains rich with specialized vocabulary. In this work, we domain adapt embeddi…
Evaluation of RAG Metrics for Question Answering in the Telecom Domain
Sujoy Roychowdhury, Sumit Soman, H G Ranjani +3
Retrieval Augmented Generation (RAG) is widely used to enable Large Language Models (LLMs) perform Question Answering (QA) tasks in various domains. However, RAG based on open-sour…