5 citations · 6 across the 12 of their papers we have counts for
12 papers
What Tokens are Learned when Tokenization is Optimized Jointly with Language Modeling?
Saketh Reddy Vemula, Parameswari Krishnamurthy
Tokenization is a fundamental component of language modeling pipelines. Despite its importance, it is often fixed, even though it significantly impacts model performance across lan…
SurakshaEval: An Indic Safety Benchmark for Multilingual LLMs
Debopriyo Banerjee, Kapil Rajesh Kavitha, Angana Borah +11
Existing safety evaluation datasets for large language models (LLMs) predominantly focus on English and Western contexts, often overlooking the linguistic diversity and culturally…
Multilingual Idioms in Sentences and Conversations Across High-, Medium-, and Low-Resource Languages
Saeed Almheiri, Bilal Elbouardi, Salsabila Zahirah Pranida +16
Idiomatic expressions pose a major challenge for multilingual NLP because their meanings shift between figurative and literal usage, often requiring context for accurate interpreta…
Typologically-Informed Candidate Reranking for LLM-based Translation into Low-Resource Languages
Nipuna Abeykoon, Ashen Weerathunga, Pubudu Wijesinghe +1
Large language models trained predominantly on high-resource languages exhibit systematic biases toward dominant typological patterns, leading to structural non-conformance when tr…
Decoding Fake Narratives in Spreading Hateful Stories: A Dual-Head RoBERTa Model with Multi-Task Learning
Yash Bhaskar, Sankalp Bahad, Parameswari Krishnamurthy
Social media platforms, while enabling global connectivity, have become hubs for the rapid spread of harmful content, including hate speech and fake narratives \cite{davidson2017au…
Yes-MT's Submission to the Low-Resource Indic Language Translation Shared Task in WMT 2024
Yash Bhaskar, Parameswari Krishnamurthy
This paper presents the systems submitted by the Yes-MT team for the Low-Resource Indic Language Translation Shared Task at WMT 2024 (Pakray et al., 2024), focusing on translating…
CorIL: Towards Enriching Indian Language to Indian Language Parallel Corpora and Machine Translation Systems
Soham Bhattacharjee, Mukund K Roy, Yathish Poojary +19
India's linguistic landscape is one of the most diverse in the world, comprising over 120 major languages and approximately 1,600 additional languages, with 22 officially recognize…
Rethinking Tokenization for Rich Morphology: The Dominance of Unigram over BPE and Morphological Alignment
Saketh Reddy Vemula, Sandipan Dandapat, Dipti Misra Sharma +1
The relationship between tokenizer algorithm (e.g., Byte-Pair Encoding (BPE), Unigram), morphological alignment, tokenization quality (e.g., compression efficiency), and downstream…
keepitsimple at SemEval-2025 Task 3: LLM-Uncertainty based Approach for Multilingual Hallucination Span Detection
Saketh Reddy Vemula, Parameswari Krishnamurthy
Identification of hallucination spans in black-box language model generated text is essential for applications in the real world. A recent attempt at this direction is SemEval-2025…
Fine-tuning Pre-trained Named Entity Recognition Models For Indian Languages
Sankalp Bahad, Pruthwik Mishra, Karunesh Arora +3
Named Entity Recognition (NER) is a useful component in Natural Language Processing (NLP) applications. It is used in various tasks such as Machine Translation, Summarization, Info…
Exploring News Summarization and Enrichment in a Highly Resource-Scarce Indian Language: A Case Study of Mizo
Abhinaba Bala, Ashok Urlana, Rahul Mishra +1
Obtaining sufficient information in one's mother tongue is crucial for satisfying the information needs of the users. While high-resource languages have abundant online resources,…
Assessing Translation capabilities of Large Language Models involving English and Indian Languages
Vandan Mujadia, Ashok Urlana, Yash Bhaskar +4
Generative Large Language Models (LLMs) have achieved remarkable advancements in various NLP tasks. In this work, our aim is to explore the multilingual capabilities of large langu…