From the 2 of 15 linked papers with an AI index.
15 papers
Translating Classical Poetry into Modern Prose
Chalamalasetti Kranti, Sowmya Vajjala
The paper presents Padyam2Gadyam, a dataset of 13th‑17th century Telugu classical poems paired with human‑verified modern Telugu and English prose translations, and evaluates machi…
LLM Judges Can Be Too Generous When There Is No Reference Answer
Chalamalasetti Kranti, Sowmya Vajjala
The paper studies how large language model judges evaluate open-ended responses without reference answers, showing they often over-credit incorrect answers and that adding referenc…
CommonLID: Re-evaluating State-of-the-Art Language Identification Performance on Web Data
Pedro Ortiz Suarez, Laurie Burchell, Catherine Arnett +94
Language identification (LID) is a fundamental step in curating multilingual corpora. However, LID models still perform poorly for many languages, especially on the noisy and heter…
ComplexityMT: Benchmarking the Interaction Between Text Complexity and Machine Translation
Joseph Marvin Imperial, Junhong Liang, Belal Shoer +9
When a text is translated, does the translation retain the complexity of the original? We introduce ComplexityMT, a new challenge for assessing how text complexity and machine tran…
MATA: Mindful Assessment of the Telugu Abilities of Large Language Models
Chalamalasetti Kranti, Sowmya Vajjala
In this paper, we introduce MATA, a novel evaluation dataset to assess the ability of Large Language Models (LLMs) in Telugu language, comprising 729 carefully curated multiple-cho…
Improving Methodologies for LLM Evaluations Across Global Languages
Akriti Vij, Benjamin Chua, Darshini Ramiah +43
As frontier AI models are deployed globally, it is essential that their behaviour remains safe and reliable across diverse linguistic and cultural contexts. To examine how current…