5 citations · 6 across the 12 of their papers we have counts for
4 papers · 1 filter
What Tokens are Learned when Tokenization is Optimized Jointly with Language Modeling?
Saketh Reddy Vemula, Parameswari Krishnamurthy
Tokenization is a fundamental component of language modeling pipelines. Despite its importance, it is often fixed, even though it significantly impacts model performance across lan…
SurakshaEval: An Indic Safety Benchmark for Multilingual LLMs
Debopriyo Banerjee, Kapil Rajesh Kavitha, Angana Borah +11
Existing safety evaluation datasets for large language models (LLMs) predominantly focus on English and Western contexts, often overlooking the linguistic diversity and culturally…
Multilingual Idioms in Sentences and Conversations Across High-, Medium-, and Low-Resource Languages
Saeed Almheiri, Bilal Elbouardi, Salsabila Zahirah Pranida +16
Idiomatic expressions pose a major challenge for multilingual NLP because their meanings shift between figurative and literal usage, often requiring context for accurate interpreta…
Typologically-Informed Candidate Reranking for LLM-based Translation into Low-Resource Languages
Nipuna Abeykoon, Ashen Weerathunga, Pubudu Wijesinghe +1
Large language models trained predominantly on high-resource languages exhibit systematic biases toward dominant typological patterns, leading to structural non-conformance when tr…