6 papers · 1 filter
EnSiTa - A Trilingual Multi-Domain Parallel Dataset and Benchmark for Domain-Specific Machine Translation
Surangika Ranathunga, Nisansa de Silva, Aloka Fernando +8
Machine Translation (MT) for low-resource languages remains far behind that of high-resource languages, and the gap is widest in specialised domains, where parallel data is scarce…
grapheme-kit: Grapheme-Level Metrics and Text Processing for Multilingual NLP
Izzath Nisfer, Ashini Kavindya, Ovindu Atukorala +2
Existing lexical distance, similarity, and evaluation metrics operate on Unicode code points, which can misrepresent errors in writing systems where a single grapheme is represente…
emb-diversity: A Tool for Embedding-Based Measurement of Data Diversity
Cantao Su, Menan Velayuthan, Esther Ploeger +2
There is growing evidence that data diversity is crucial for developing fair and robust NLP models. However, current approaches to measure diversity remain inconsistent and fragmen…
From Phonemes to Meaning: Evaluating Large Language Models on Tamil
Jeyarajalingam Varsha, Menan Velayuthan, Sumirtha Karunakaran +2
Large Language Models (LLMs) have shown strong generalization across tasks in high-resource languages; however, their linguistic competence in low-resource and morphologically rich…
Global PIQA: Evaluating Commonsense Reasoning Across 100+ Languages and Cultures
Tyler A. Chang, Catherine Arnett, Abdelrahman Sadallah +377
To date, there exist almost no culturally-specific evaluation benchmarks for large language models (LLMs) that cover a large number of languages and cultures. In this paper, we pre…
Egalitarian Language Representation in Language Models: It All Begins with Tokenizers
Menan Velayuthan, Kengatharaiyer Sarveswaran
Tokenizers act as a bridge between human language and the latent space of language models, influencing how language is represented in these models. Due to the immense popularity of…