Nanbeige4-3B Technical Report: Exploring the Frontier of Small Language Models

Chen Yang, Guangyue Peng, Jiaying Zhu +16

We present Nanbeige4-3B, a family of small-scale but high-performing language models. Pretrained on 23T high-quality tokens and finetuned on over 30 million diverse instructions, w…