4 citations · 9 across the 4 of their papers we have counts for
3 papers · 1 filter
Continual Learning Under Language Shift
Evangelia Gogoulou, Timothée Lesort, Magnus Boman +1
The recent increase in data and model scale for language model pre-training has led to huge training costs. In scenarios where new data become available over time, updating a model…
GPT-SW3: An Autoregressive Language Model for the Nordic Languages
Ariel Ekgren, Amaru Cuba Gyllensten, Felix Stollenwerk +7
This paper details the process of developing the first native large generative language model for the Nordic languages, GPT-SW3. We cover all parts of the development process, from…
The Nordic Pile: A 1.2TB Nordic Dataset for Language Modeling
Joey Öhman, Severine Verlinden, Ariel Ekgren +5
Pre-training Large Language Models (LLMs) require massive amounts of text data, and the performance of the LLMs typically correlates with the scale and quality of the datasets. Thi…