78 citations · 226 across the 23 of their papers we have counts for
26 papers
Are Large Language Model-based Evaluators the Solution to Scaling Up Multilingual Evaluation?
Rishav Hada, Varun Gumma, Adrian de Wynter +5
Large Language Models (LLMs) excel in various Natural Language Processing (NLP) tasks, yet their evaluation, particularly in languages beyond the top , remains inadequate due t…
On Evaluating and Mitigating Gender Biases in Multilingual Settings
Aniket Vashishtha, Kabir Ahuja, Sunayana Sitaram
While understanding and removing gender biases in language models has been a long-standing problem in Natural Language Processing, prior research work has primarily been limited to…
Bridging the Language Gap: Dynamic Learning Strategies for Improving Multilingual Performance in LLMs
Somnath Kumar, Vaibhav Balloli, Mercy Ranjit +5
Large language models (LLMs) have revolutionized various domains but still struggle with non-Latin scripts and low-resource languages. This paper addresses the critical challenge o…
DiTTO: A Feature Representation Imitation Approach for Improving Cross-Lingual Transfer
Shanu Kumar, Abbaraju Soujanya, Sandipan Dandapat +2
Zero-shot cross-lingual transfer is promising, however has been shown to be sub-optimal, with inferior transfer performance across low-resource languages. In this work, we envision…
MEGA: Multilingual Evaluation of Generative AI
Kabir Ahuja, Harshita Diddee, Rishav Hada +9
Generative AI models have shown impressive performance on many Natural Language Processing tasks such as language understanding, reasoning, and language generation. An important qu…
Analysing the Masked predictive coding training criterion for pre-training a Speech Representation Model
Hemant Yadav, Sunayana Sitaram, Rajiv Ratn Shah
Recent developments in pre-trained speech representation utilizing self-supervised learning (SSL) have yielded exceptional results on a variety of downstream tasks. One such techni…
Fairness in Language Models Beyond English: Gaps and Challenges
Krithika Ramesh, Sunayana Sitaram, Monojit Choudhury
With language models becoming increasingly ubiquitous, it has become essential to address their inequitable treatment of diverse demographic groups and factors. Most research on ev…
A Survey of Code-switching: Linguistic and Social Perspectives for Language Technologies
A. Seza Doğruöz, Sunayana Sitaram, Barbara E. Bullock +1
The analysis of data in which multiple languages are represented has gained popularity among computational linguists in recent years. So far, much of this research focuses mainly o…
Benchmarking Evaluation Metrics for Code-Switching Automatic Speech Recognition
Injy Hamed, Amir Hussein, Oumnia Chellah +5
Code-switching poses a number of challenges and opportunities for multilingual automatic speech recognition. In this paper, we focus on the question of robust and fair evaluation m…
On the Calibration of Massively Multilingual Language Models
Kabir Ahuja, Sunayana Sitaram, Sandipan Dandapat +1
Massively Multilingual Language Models (MMLMs) have recently gained popularity due to their surprising effectiveness in cross-lingual transfer. While there has been much work in ev…
Beyond Static Models and Test Sets: Benchmarking the Potential of Pre-trained Models Across Tasks and Languages
Kabir Ahuja, Sandipan Dandapat, Sunayana Sitaram +1
Although recent Massively Multilingual Language Models (MMLMs) like mBERT and XLMR support around 100 languages, most existing multilingual NLP benchmarks provide evaluation data i…
Multilingual CheckList: Generation and Evaluation
Karthikeyan K, Shaily Bhatt, Pankaj Singh +4
Multilingual evaluation benchmarks usually contain limited high-resource languages and do not test models for specific linguistic capabilities. CheckList is a template-based evalua…
A Survey of Multilingual Models for Automatic Speech Recognition
Hemant Yadav, Sunayana Sitaram
Although Automatic Speech Recognition (ASR) systems have achieved human-like performance for a few languages, the majority of the world's languages do not have usable systems due t…
Predicting the Performance of Multilingual NLP Models
Anirudh Srinivasan, Sunayana Sitaram, Tanuja Ganu +3
Recent advancements in NLP have given us models like mBERT and XLMR that can serve over 100 languages. The languages that these models are evaluated on, however, are very few in nu…
On the Universality of Deep Contextual Language Models
Shaily Bhatt, Poonam Goyal, Sandipan Dandapat +2
Deep Contextual Language Models (LMs) like ELMO, BERT, and their successors dominate the landscape of Natural Language Processing due to their ability to scale across multiple task…
Multilingual and code-switching ASR challenges for low resource Indian languages
Anuj Diwan, Rakesh Vaideeswaran, Sanket Shah +19
Recently, there is increasing interest in multilingual automatic speech recognition (ASR) where a speech recognition system caters to multiple low resource languages by taking adva…
Cross-lingual and Multilingual Spoken Term Detection for Low-Resource Indian Languages
Sanket Shah, Satarupa Guha, Simran Khanuja +1
Spoken Term Detection (STD) is the task of searching for words or phrases within audio, given either text or spoken input as a query. In this work, we use state-of-the-art Hindi, T…
mask-Net: Learning Context Aware Invariant Features using Adversarial Forgetting (Student Abstract)
Hemant Yadav, Atul Anshuman Singh, Rachit Mittal +3
Training a robust system, e.g.,Speech to Text (STT), requires large datasets. Variability present in the dataset such as unwanted nuisances and biases are the reason for the need o…
Learning not to Discriminate: Task Agnostic Learning for Improving Monolingual and Code-switched Speech Recognition
Gurunath Reddy Madhumani, Sanket Shah, Basil Abraham +2
Recognizing code-switched speech is challenging for Automatic Speech Recognition (ASR) for a variety of reasons, including the lack of code-switched training data. Recently, we sho…
Learning to Recognize Code-switched Speech Without Forgetting Monolingual Speech Recognition
Sanket Shah, Basil Abraham, Gurunath Reddy M +2
Recently, there has been significant progress made in Automatic Speech Recognition (ASR) of code-switched speech, leading to gains in accuracy on code-switched datasets in many lan…
GLUECoS : An Evaluation Benchmark for Code-Switched NLP
Simran Khanuja, Sandipan Dandapat, Anirudh Srinivasan +2
Code-switching is the use of more than one language in the same conversation or utterance. Recently, multilingual contextual embedding models, trained on multiple monolingual corpo…
A New Dataset for Natural Language Inference from Code-mixed Conversations
Simran Khanuja, Sandipan Dandapat, Sunayana Sitaram +1
Natural Language Inference (NLI) is the task of inferring the logical relationship, typically entailment or contradiction, between a premise and hypothesis. Code-mixing is the use…
Unsung Challenges of Building and Deploying Language Technologies for Low Resource Language Communities
Pratik Joshi, Christain Barnes, Sebastin Santy +7
In this paper, we examine and analyze the challenges associated with developing and introducing language technologies to low-resource language communities. While doing so, we bring…
End-to-End ASR for Code-switched Hindi-English Speech
Brij Mohan Lal Srivastava, Basil Abraham, Sunayana Sitaram +2
End-to-end (E2E) models have been explored for large speech corpora and have been found to match or outperform traditional pipeline-based systems in some languages. However, most p…