OpenLanguageModel: Readable and Composable Small-Language-Model Pretraining for Education and Research

Tavish Mankash, Vardhaman Kalloli, Keshava Prasad +1

OpenLanguageModel (OLM) is an open-source PyTorch library for building and pretraining small language models while keeping their machinery visible. In OLM, model code reads like th…