3 papers
cs.CL2025
From Phonemes to Meaning: Evaluating Large Language Models on Tamil
Jeyarajalingam Varsha, Menan Velayuthan, Sumirtha Karunakaran +2
Large Language Models (LLMs) have shown strong generalization across tasks in high-resource languages; however, their linguistic competence in low-resource and morphologically rich…
cs.CL2024
Building Tamil Treebanks
Kengatharaiyer Sarveswaran
Treebanks are important linguistic resources, which are structured and annotated corpora with rich linguistic annotations. These resources are used in Natural Language Processing (…
cs.CL2024
Egalitarian Language Representation in Language Models: It All Begins with Tokenizers
Menan Velayuthan, Kengatharaiyer Sarveswaran
Tokenizers act as a bridge between human language and the latent space of language models, influencing how language is represented in these models. Due to the immense popularity of…