5 papers · 1 filter
Advancing Polish Language Modeling through Tokenizer Optimization in the Bielik v3 7B and 11B Series
Krzysztof Ociepa, Łukasz Flis, Remigiusz Kinas +2
The development of the Bielik v3 PL series, encompassing both the 7B and 11B parameter variants, represents a significant milestone in the field of language-specific large language…
Bielik-Minitron-7B: Compressing Large Language Models via Structured Pruning and Knowledge Distillation for the Polish Language
Remigiusz Kinas, Paweł Kiszczak, Sergio P. Perez +4
This report details the creation of Bielik-Minitron-7B, a compressed 7.35B parameter version of the Bielik-11B-v3.0 model, specifically optimized for European languages. By leverag…
Bielik 11B v3: Multilingual Large Language Model for European Languages
Krzysztof Ociepa, Łukasz Flis, Remigiusz Kinas +2
We present Bielik 11B v3, a state-of-the-art language model highly optimized for the Polish language, while also maintaining strong capabilities in other European languages. This m…
Bielik 11B v2 Technical Report
Krzysztof Ociepa, Łukasz Flis, Krzysztof Wróbel +2
We present Bielik 11B v2, a state-of-the-art language model optimized for Polish text processing. Built on the Mistral 7B v0.2 architecture and scaled to 11B parameters using depth…
Bielik 7B v0.1: A Polish Language Model -- Development, Insights, and Evaluation
Krzysztof Ociepa, Łukasz Flis, Krzysztof Wróbel +2
We introduce Bielik 7B v0.1, a 7-billion-parameter generative text model for Polish language processing. Trained on curated Polish corpora, this model addresses key challenges in l…