17 papers
Articulatory Source-Filter TTS: Physically Grounded Control through Vocal Tract Kinematics
Jesuraj Bandekar, Shinji Watanabe, Prasanta Kumar Ghosh
Modern neural text-to-speech (TTS) systems achieve remarkable acoustic fidelity but act as black boxes, offering little interpretable control over the vocal tract filter. We propos…
A Training Criterion with Token-Level Tolerance to Transcription Ambiguity for Automatic Speech Recognition
Saurabh Kumar, Diptiman Mohanta, Prasanta Kumar Ghosh
Automatic speech recognition is typically trained assuming that the reference transcript is the only valid labeling of an utterance, yet even nominally verbatim transcripts contain…
VAANI Noise Event Dataset: A curated spontaneous speech dataset annotated with timestamps for noise events
Pavan Kumar J, Agneedh Basu, Pranav Bhat +3
Most public sound-event corpora are optimized either for general audio tagging or for clean speech separation, and comparatively few provide strong timestamped noise annotations la…
SraVaani 1.0: Scaling Inclusive Speech Recognition for Indic Languages
Sujith Pulikodan, Agneedh Basu, Pavan Kumar J +4
India's linguistic landscape spans over 700 languages and thousands of dialects, yet the vast majority of automatic speech recognition (ASR) systems support only a small fraction o…
Jointly Improving Dialect Identification and ASR in Indian Languages using Multimodal Feature Fusion
Saurabh Kumar, Amartyaveer, Prasanta Kumar Ghosh
Automatic Speech Recognition (ASR) and Dialect Identification (DID) are crucial for Indian languages, many of which are low-resource and exhibit significant dialectal differences.…
Enhancing Acoustic-to-Articulatory Inversion with Multi-Target Pretraining for Low-Resource Settings
Jesuraj Bandekar, Prasanta Kumar Ghosh
Acoustic-to-Articulatory Inversion (AAI) estimates vocal tract articulator movements from speech, benefiting tasks like ASR, speech synthesis, and speaker verification. While deep…
Audio--Image Alignment as a Continued-Pretraining Stage in Automatic Speech Recognition
Sujith Pulikodan, Nihar Desai, Prasanta Kumar Ghosh
Thousands of languages are spoken worldwide, yet many remain under-resourced for Automatic Speech Recognition (ASR) due to the limited availability of high-quality transcribed spee…
Vaani Benchmark V1.0: An Inclusive Multimodal Benchmark Dataset for Hindi
Sujith Pulikodan, Agneedh Basu, Saurabh Kumar +5
Benchmarking is critical for the systematic evaluation of machine learning systems. While several open-source datasets are available for Hindi, existing benchmarks remain limited i…
Analyzing Language and Geographical Variation in Speech Representations Across 60 Indic Languages
Pavan Kumar J, Agneedh Basu, Pranav Bhat +4
Self-supervised speech encoders are often fine-tuned with language supervision, which can overlook geographical variation. To understand the learned representations under joint sup…
An Analysis of the Effectiveness of Synthetic Speech Data for ASR Fine-tuning in Selected Indic Languages
Sujith Pulikodan, Agneedh Basu, Pavan Kumar +4
Synthetic data has the potential to be a valuable resource for training machine learning models, particularly Automatic Speech Recognition (ASR) Systems; however, its effectiveness…
Factors affecting ASR performance: A study using state of the art ASR models in Indic Languages
Agneedh Basu, Pavan Kumar J, Pranav Bhat +4
ASR performance varies across languages, speakers, and recording conditions, yet systematic analysis for Indic languages remain limited. We present a large-scale study of decoded o…
A Comparative Study of Pre-trained Speech Encoders and Training Objectives for Large-Scale Indic Spoken Language Identification
Agneedh Basu, Pavan Kumar J, Sujith P +3
Spoken language identification (LID) for Indian languages is a challenging problem due to the large number of languages, significant phonetic overlap among related varieties, and t…
Bottleneck Transformer-Based Approach for Improved Automatic STOI Score Prediction
Amartyaveer, Murali Kadambi, Chandra Mohan Sharma +2
In this study, we have presented a novel approach to predict the Short-Time Objective Intelligibility (STOI) metric using a bottleneck transformer architecture. Traditional methods…
VAANI: Capturing the language landscape for an inclusive digital India
Sujith Pulikodan, Abhayjeet Singh, Agneedh Basu +17
Voice based technologies have the potential to bridge digital accessibility gaps; however, existing datasets fail to capture the linguistic and regional diversity of Indic language…
Discovering phoneme-specific critical articulators through a data-driven approach
Jesuraj Bandekar, Sathvik Udupa, Prasanta Kumar Ghosh
We propose an approach for learning critical articulators for phonemes through a machine learning approach. We formulate the learning with three models trained end to end. First, w…
SPIRE-SIES: A Spontaneous Indian English Speech Corpus
Abhayjeet Singh, Charu Shah, Rajashri Varadaraj +2
In this paper, we present a 170.83 hour Indian English spontaneous speech dataset. Lack of Indian English speech data is one of the major hindrances in developing robust speech sys…
Speaking rate attention-based duration prediction for speed control TTS
Jesuraj Bandekar, Sathvik Udupa, Abhayjeet Singh +6
With the advent of high-quality speech synthesis, there is a lot of interest in controlling various prosodic attributes of speech. Speaking rate is an essential attribute towards m…