78 citations · 226 across the 23 of their papers we have counts for
20 papers · 1 filter
Are Large Language Model-based Evaluators the Solution to Scaling Up Multilingual Evaluation?
Rishav Hada, Varun Gumma, Adrian de Wynter +5
Large Language Models (LLMs) excel in various Natural Language Processing (NLP) tasks, yet their evaluation, particularly in languages beyond the top , remains inadequate due t…
On Evaluating and Mitigating Gender Biases in Multilingual Settings
Aniket Vashishtha, Kabir Ahuja, Sunayana Sitaram
While understanding and removing gender biases in language models has been a long-standing problem in Natural Language Processing, prior research work has primarily been limited to…
Bridging the Language Gap: Dynamic Learning Strategies for Improving Multilingual Performance in LLMs
Somnath Kumar, Vaibhav Balloli, Mercy Ranjit +5
Large language models (LLMs) have revolutionized various domains but still struggle with non-Latin scripts and low-resource languages. This paper addresses the critical challenge o…
DiTTO: A Feature Representation Imitation Approach for Improving Cross-Lingual Transfer
Shanu Kumar, Abbaraju Soujanya, Sandipan Dandapat +2
Zero-shot cross-lingual transfer is promising, however has been shown to be sub-optimal, with inferior transfer performance across low-resource languages. In this work, we envision…
MEGA: Multilingual Evaluation of Generative AI
Kabir Ahuja, Harshita Diddee, Rishav Hada +9
Generative AI models have shown impressive performance on many Natural Language Processing tasks such as language understanding, reasoning, and language generation. An important qu…
Fairness in Language Models Beyond English: Gaps and Challenges
Krithika Ramesh, Sunayana Sitaram, Monojit Choudhury
With language models becoming increasingly ubiquitous, it has become essential to address their inequitable treatment of diverse demographic groups and factors. Most research on ev…
A Survey of Code-switching: Linguistic and Social Perspectives for Language Technologies
A. Seza Doğruöz, Sunayana Sitaram, Barbara E. Bullock +1
The analysis of data in which multiple languages are represented has gained popularity among computational linguists in recent years. So far, much of this research focuses mainly o…
On the Calibration of Massively Multilingual Language Models
Kabir Ahuja, Sunayana Sitaram, Sandipan Dandapat +1
Massively Multilingual Language Models (MMLMs) have recently gained popularity due to their surprising effectiveness in cross-lingual transfer. While there has been much work in ev…
Beyond Static Models and Test Sets: Benchmarking the Potential of Pre-trained Models Across Tasks and Languages
Kabir Ahuja, Sandipan Dandapat, Sunayana Sitaram +1
Although recent Massively Multilingual Language Models (MMLMs) like mBERT and XLMR support around 100 languages, most existing multilingual NLP benchmarks provide evaluation data i…
Multilingual CheckList: Generation and Evaluation
Karthikeyan K, Shaily Bhatt, Pankaj Singh +4
Multilingual evaluation benchmarks usually contain limited high-resource languages and do not test models for specific linguistic capabilities. CheckList is a template-based evalua…
A Survey of Multilingual Models for Automatic Speech Recognition
Hemant Yadav, Sunayana Sitaram
Although Automatic Speech Recognition (ASR) systems have achieved human-like performance for a few languages, the majority of the world's languages do not have usable systems due t…
Predicting the Performance of Multilingual NLP Models
Anirudh Srinivasan, Sunayana Sitaram, Tanuja Ganu +3
Recent advancements in NLP have given us models like mBERT and XLMR that can serve over 100 languages. The languages that these models are evaluated on, however, are very few in nu…
On the Universality of Deep Contextual Language Models
Shaily Bhatt, Poonam Goyal, Sandipan Dandapat +2
Deep Contextual Language Models (LMs) like ELMO, BERT, and their successors dominate the landscape of Natural Language Processing due to their ability to scale across multiple task…
Multilingual and code-switching ASR challenges for low resource Indian languages
Anuj Diwan, Rakesh Vaideeswaran, Sanket Shah +19
Recently, there is increasing interest in multilingual automatic speech recognition (ASR) where a speech recognition system caters to multiple low resource languages by taking adva…
Cross-lingual and Multilingual Spoken Term Detection for Low-Resource Indian Languages
Sanket Shah, Satarupa Guha, Simran Khanuja +1
Spoken Term Detection (STD) is the task of searching for words or phrases within audio, given either text or spoken input as a query. In this work, we use state-of-the-art Hindi, T…
GLUECoS : An Evaluation Benchmark for Code-Switched NLP
Simran Khanuja, Sandipan Dandapat, Anirudh Srinivasan +2
Code-switching is the use of more than one language in the same conversation or utterance. Recently, multilingual contextual embedding models, trained on multiple monolingual corpo…
A New Dataset for Natural Language Inference from Code-mixed Conversations
Simran Khanuja, Sandipan Dandapat, Sunayana Sitaram +1
Natural Language Inference (NLI) is the task of inferring the logical relationship, typically entailment or contradiction, between a premise and hypothesis. Code-mixing is the use…
Unsung Challenges of Building and Deploying Language Technologies for Low Resource Language Communities
Pratik Joshi, Christain Barnes, Sebastin Santy +7
In this paper, we examine and analyze the challenges associated with developing and introducing language technologies to low-resource language communities. While doing so, we bring…
A Survey of Code-switched Speech and Language Processing
Sunayana Sitaram, Khyathi Raghavi Chandu, Sai Krishna Rallabandi +1
Code-switching, the alternation of languages within a conversation or utterance, is a common communicative phenomenon that occurs in multilingual communities across the world. This…
Polyglot Neural Language Models: A Case Study in Cross-Lingual Phonetic Representation Learning
Yulia Tsvetkov, Sunayana Sitaram, Manaal Faruqui +6
We introduce polyglot language models, recurrent neural network models trained to predict symbol sequences in many different languages using shared representations of symbols and c…