4 papers · 1 filter
PARAMANU-GANITA: Can Small Math Language Models Rival with Large Language Models on Mathematical Reasoning?
Mitodru Niyogi, Arnab Bhattacharya
In this paper, we study whether domain specific pretraining of small generative language models (SLM) from scratch with domain specialized tokenizer and Chain-of-Thought (CoT) inst…
Ayn: A Tiny yet Competitive Indian Legal Language Model Pretrained from Scratch
Mitodru Niyogi, Eric Gaussier, Arnab Bhattacharya
Decoder-only Large Language Models (LLMs) are currently the model of choice for many Natural Language Processing (NLP) applications. Through instruction fine-tuning and prompting a…
Neural Models for Source Code Synthesis and Completion
Mitodru Niyogi
Natural language (NL) to code suggestion systems assist developers in Integrated Development Environments (IDEs) by translating NL utterances into compilable code snippet. The curr…
Paramanu: Compact and Competitive Monolingual Language Models for Low-Resource Morphologically Rich Indian Languages
Mitodru Niyogi, Eric Gaussier, Arnab Bhattacharya
Multilingual large language models (LLMs) are expensive to pretrain and often suffer from imbalances across languages and datasets, English-centric bias, tokenizer oversegmentation…