2 papers
cs.CL2026
MUTANT: A Recipe for Multilingual Tokenizer Design
Souvik Rana, Arul Menezes, Ashish Kulkarni +2
Tokenizers play a crucial role in determining the performance, training efficiency, and the inference cost of Large Language Models (LLMs). Designing effective tokenizers for multi…
cs.CL2025
BhashaKritika: Building Synthetic Pretraining Data at Scale for Indic Languages
Guduru Manoj, Neel Prabhanjan Rachamalla, Ashish Kulkarni +8
In the context of pretraining of Large Language Models (LLMs), synthetic data has emerged as an alternative for generating high-quality pretraining data at scale. This is particula…