2 citations · 3 across the 12 of their papers we have counts for
14 papers · 1 filter
There is No Theoretical Curse of Multilinguality For Embedding Space Structure
Niyati Bafna, Neha Verma, Vilém Zouhar +2
A central goal of multilingual NLP is to achieve high monolingual performance per language and cross-lingual alignment for large-scale language coverage with a multilingual model.…
Last Translation Benchmark
Vilém Zouhar, Niyati Bafna, Mukund Choudhary +263
For scientific progress, we need benchmarks that test the limits of state-of-the-art models, and evaluation methods that inform us about failure cases. As models get stronger, stan…
Cultivar: A Contrastive and Locale-Oriented Translation Benchmark for Investigating Contamination and Localisation Robustness
Pinzhen Chen, Koel Dutta Chowdhury, Xiaoya Xu +20
Multilingual translation benchmarks are typically sourced in English and translated into other languages, treating language pairs as the unit of evaluation---a design that is prone…
Multilingual Reasoning Cascades Need More Context
Arnav Mazumder, Dengjia Zhang, Shuyue Stella Li +2
Translation cascades for reasoning translate the query from another language to English, reason in English, and translate the answer back to the original language. This is a compet…
Rashid: A Cipher-Based Framework for Exploring In-Context Language Learning
Niyati Bafna, Ryan Soh-Eun Shim, Barbara Plank +2
Where there is growing interest in in-context language learning (ICLL) for unseen languages with large language models, such languages usually suffer from the lack of NLP tools, da…
Omnilingual MT: Machine Translation for 1,600 Languages
Omnilingual MT Team, Belen Alastruey, Niyati Bafna +29
High-quality machine translation (MT) can scale to hundreds of languages, setting a high bar for multilingual systems. However, compared to the world's 7,000 languages, current sys…
ChiKhaPo: A Large-Scale Multilingual Benchmark for Evaluating Lexical Comprehension and Generation in Large Language Models
Emily Chang, Niyati Bafna
Existing benchmarks for large language models (LLMs) are largely restricted to high- or mid-resource languages, and often evaluate performance on higher-order tasks in reasoning an…
How Important is `Perfect' English for Machine Translation Prompts?
Patrícia Schmidtová, Niyati Bafna, Seth Aycock +4
Large language models (LLMs) have achieved top results in recent machine translation evaluations, but they are also known to be sensitive to errors and perturbations in their promp…
LID Models are Actually Accent Classifiers: Implications and Solutions for LID on Accented Speech
Niyati Bafna, Matthew Wiesner
Prior research indicates that LID model performance significantly declines on accented speech; however, the specific causes, extent, and characterization of these errors remain und…
The Translation Barrier Hypothesis: Multilingual Generation with Large Language Models Suffers from Implicit Translation Failure
Niyati Bafna, Tianjian Li, Kenton Murray +4
Multilingual generation with large language models (LLMs) is often of poor quality for mid- to low-resource languages, but the causes for this are not well-understood. We first dem…
DialUp! Modeling the Language Continuum by Adapting Models to Dialects and Dialects to Models
Niyati Bafna, Emily Chang, Nathaniel R. Robinson +4
Most of the world's languages and dialects are low-resource, and lack support in mainstream machine translation (MT) models. However, many of them have a closely-related high-resou…
Evaluating Large Language Models along Dimensions of Language Variation: A Systematik Invesdigatiom uv Cross-lingual Generalization
Niyati Bafna, Kenton Murray, David Yarowsky
While large language models exhibit certain cross-lingual generalization capabilities, they suffer from performance degradation (PD) on unseen closely-related languages (CRLs) and…
Pointer-Generator Networks for Low-Resource Machine Translation: Don't Copy That!
Niyati Bafna, Philipp Koehn, David Yarowsky
While Transformer-based neural machine translation (NMT) is very effective in high-resource settings, many languages lack the necessary large parallel corpora to benefit from it. I…
When your Cousin has the Right Connections: Unsupervised Bilingual Lexicon Induction for Related Data-Imbalanced Languages
Niyati Bafna, Cristina España-Bonet, Josef van Genabith +2
Most existing approaches for unsupervised bilingual lexicon induction (BLI) depend on good quality static or contextual embeddings requiring large monolingual corpora for both lang…