6 papers
MUTANT: A Recipe for Multilingual Tokenizer Design
Souvik Rana, Arul Menezes, Ashish Kulkarni +2
Tokenizers play a crucial role in determining the performance, training efficiency, and the inference cost of Large Language Models (LLMs). Designing effective tokenizers for multi…
Designing Production-Scale OCR for India: Multilingual and Domain-Specific Systems
Ali Faraz, Raja Kolla, Ashish Kulkarni +1
Designing Optical Character Recognition (OCR) systems for India requires balancing linguistic diversity, document heterogeneity, and deployment constraints. In this paper, we study…
VoiceAgentBench: Are Voice Assistants ready for agentic tasks?
Dhruv Jain, Harshit Shukla, Gautam Rajeev +3
Large scale Speech Language Models have enabled voice assistants capable of understanding natural spoken queries and performing complex tasks. However, existing speech benchmarks l…
BhashaKritika: Building Synthetic Pretraining Data at Scale for Indic Languages
Guduru Manoj, Neel Prabhanjan Rachamalla, Ashish Kulkarni +8
In the context of pretraining of Large Language Models (LLMs), synthetic data has emerged as an alternative for generating high-quality pretraining data at scale. This is particula…
Pragyaan: Designing and Curating High-Quality Cultural Post-Training Datasets for Indian Languages
Neel Prabhanjan Rachamalla, Aravind Konakalla, Gautam Rajeev +3
The effectiveness of Large Language Models (LLMs) depends heavily on the availability of high-quality post-training data, particularly instruction-tuning and preference-based examp…
Chitranuvad: Adapting Multi-Lingual LLMs for Multimodal Translation
Shaharukh Khan, Ayush Tarun, Ali Faraz +7
In this work, we provide the system description of our submission as part of the English to Lowres Multimodal Translation Task at the Workshop on Asian Translation (WAT2024). We in…