326 citations · 717 across the 90 of their papers we have counts for
78 papers · 1 filter
Vimarsha: Faithful ASR Evaluation for Indian Languages with Demographic Diversity, In-the-Wild Audio and Spelling Variations
Kaushal Santosh Bhogale, Srija Anand, Sadakopa Ramakrishnan Thothathiri +3
Evaluation benchmarks for Indian language automatic speech recognition (ASR) suffer from two systematic biases: optimistic scores from clean, controlled audio conditions, and pessi…
IndicQE-APE: A Consolidated Benchmark for Quality Estimation and Automatic Post-Editing for Indic Languages
Diptesh Kanojia, Archchana Sindhujan, Sourabh Deoghare +14
Indic quality estimation (QE) and automatic post-editing (APE) data is spread across separate releases, so no single resource supports training and evaluation across tasks and lang…
Indic DiarBench: A Multilingual Joint Diarization and ASR Benchmark for Indian Languages
Deovrat Mehendale, Aditya Mehndiratta, Dhruv Rathi +2
In this work, we introduce Indic DiarBench, a speaker diarization and ASR benchmark dataset spanning all 22 scheduled languages of India. This corpus comprises approximately 108 ho…
Preferences of a Voice-First Nation: Large-Scale Pairwise Evaluation and Preference Analysis for TTS in Indian Languages
Srija Anand, Ashwin Sankar, Ishvinder Sethi +10
Crowdsourced pairwise evaluation has emerged as a scalable approach for assessing foundation models. However, applying it to Text to Speech(TTS) introduces high variance due to lin…
Voice of India: A Large-Scale Benchmark for Real-World Speech Recognition in India
Kaushal Bhogale, Manas Dhir, Amritansh Walecha +11
Existing Indic ASR benchmarks often use scripted, clean speech and leaderboard driven evaluation that encourages dataset specific overfitting. In addition, strict single reference…
Towards Orthographically-Informed Evaluation of Speech Recognition Systems for Indian Languages
Kaushal Santosh Bhogale, Tahir Javed, Greeshma Susan John +4
Evaluating ASR systems for Indian languages is challenging due to spelling variations, suffix splitting flexibility, and non-standard spellings in code-mixed words. Traditional Wor…
The State Of TTS: A Case Study with Human Fooling Rates
Praveen Srinivasa Varadhan, Sherry Thomas, Sai Teja M. S. +2
While subjective evaluations in recent years indicate rapid progress in TTS, can current TTS systems truly pass a human deception test in a Turing-like evaluation? We introduce Hum…
NIRANTAR: Continual Learning with New Languages and Domains on Real-world Speech Data
Tahir Javed, Kaushal Bhogale, Mitesh M. Khapra
We introduce Nirantar, a comprehensive framework for evaluating continual learning (CL) in multilingual and multi-domain ASR. Designed to reflect real-world CL challenges, Nirantar…
FairI Tales: Evaluation of Fairness in Indian Contexts with a Focus on Bias and Stereotypes
Janki Atul Nawale, Mohammed Safi Ur Rahman Khan, Janani D +3
Existing studies on fairness are largely Western-focused, making them inadequate for culturally diverse countries such as India. To address this gap, we introduce INDIC-BIAS, a com…
Phir Hera Fairy: An English Fairytaler is a Strong Faker of Fluent Speech in Low-Resource Indian Languages
Praveen Srinivasa Varadhan, Srija Anand, Soma Siddhartha +1
What happens when an English Fairytaler is fine-tuned on Indian languages? We evaluate how the English F5-TTS model adapts to 11 Indian languages, measuring polyglot fluency, voice…
RASMALAI: Resources for Adaptive Speech Modeling in Indian Languages with Accents and Intonations
Ashwin Sankar, Yoach Lacombe, Sherry Thomas +3
We introduce RASMALAI, a large-scale speech dataset with rich text descriptions, designed to advance controllable and expressive text-to-speech (TTS) synthesis for 23 Indian langua…
RomanLens: The Role Of Latent Romanization In Multilinguality In LLMs
Alan Saji, Jaavid Aktar Husain, Thanmay Jayakumar +3
Large Language Models (LLMs) exhibit strong multilingual performance despite being predominantly trained on English-centric corpora. This raises a fundamental question: How do LLMs…
Pralekha: Cross-Lingual Document Alignment for Indic Languages
Sanjay Suryanarayanan, Haiyue Song, Mohammed Safi Ur Rahman Khan +2
Mining parallel document pairs for document-level machine translation (MT) remains challenging due to the limitations of existing Cross-Lingual Document Alignment (CLDA) techniques…
Rethinking MUSHRA: Addressing Modern Challenges in Text-to-Speech Evaluation
Praveen Srinivasa Varadhan, Amogh Gulati, Ashwin Sankar +8
Despite rapid advancements in TTS models, a consistent and robust human evaluation framework is still lacking. For example, MOS tests fail to differentiate between similar models,…
Towards Building Large Scale Datasets and State-of-the-Art Automatic Speech Translation Systems for 14 Indian Languages
Ashwin Sankar, Sparsh Jain, Nikhil Narasimhan +6
Speech translation for Indian languages remains a challenging task due to the scarcity of large-scale, publicly available datasets that capture the linguistic diversity and domain…
ELAICHI: Enhancing Low-resource TTS by Addressing Infrequent and Low-frequency Character Bigrams
Srija Anand, Praveen Srinivasa Varadhan, Mehak Singal +1
Recent advancements in Text-to-Speech (TTS) technology have led to natural-sounding speech for English, primarily due to the availability of large-scale, high-quality web data. How…
IndicVoices-R: Unlocking a Massive Multilingual Multi-speaker Speech Corpus for Scaling Indian TTS
Ashwin Sankar, Srija Anand, Praveen Srinivasa Varadhan +7
Recent advancements in text-to-speech (TTS) synthesis show that large-scale models trained with extensive web data produce highly natural-sounding output. However, such data is sca…
Cross-Lingual Auto Evaluation for Assessing Multilingual LLMs
Sumanth Doddapaneni, Mohammed Safi Ur Rahman Khan, Dilip Venkatesh +3
Evaluating machine-generated text remains a significant challenge in NLP, especially for non-English languages. Current methodologies, including automated metrics, human assessment…
Rasa: Building Expressive Speech Synthesis Systems for Indian Languages in Low-resource Settings
Praveen Srinivasa Varadhan, Ashwin Sankar, Giri Raju +1
We release Rasa, the first multilingual expressive TTS dataset for any Indian language, which contains 10 hours of neutral speech and 1-3 hours of expressive speech for each of the…
Empowering Low-Resource Language ASR via Large-Scale Pseudo Labeling
Kaushal Santosh Bhogale, Deovrat Mehendale, Niharika Parasa +4
In this study, we tackle the challenge of limited labeled data for low-resource languages in ASR, focusing on Hindi. Specifically, we explore pseudo-labeling, by proposing a generi…
LAHAJA: A Robust Multi-accent Benchmark for Evaluating Hindi ASR Systems
Tahir Javed, Janki Nawale, Sakshi Joshi +4
Hindi, one of the most spoken language of India, exhibits a diverse array of accents due to its usage among individuals from diverse linguistic origins. To enable a robust evaluati…
Enhancing Out-of-Vocabulary Performance of Indian TTS Systems for Practical Applications through Low-Effort Data Strategies
Srija Anand, Praveen Srinivasa Varadhan, Ashwin Sankar +2
Publicly available TTS datasets for low-resource languages like Hindi and Tamil typically contain 10-20 hours of data, leading to poor vocabulary coverage. This limitation becomes…
An Empirical Comparison of Vocabulary Expansion and Initialization Approaches for Language Models
Nandini Mundra, Aditya Nanda Kishore, Raj Dabre +3
Language Models (LMs) excel in natural language processing tasks for English but show reduced performance in most other languages. This problem is commonly tackled by continually p…
How Good is Zero-Shot MT Evaluation for Low Resource Indian Languages?
Anushka Singh, Ananya B. Sai, Raj Dabre +3
While machine translation evaluation has been studied primarily for high-resource languages, there has been a recent interest in evaluation for low-resource languages due to the in…