21 citations · 39 across the 8 of their papers we have counts for
9 papers
MultiSynt/MT: Trillion-Token Multi-Parallel Pre-Training Data Translated Across 36 Languages
Maximilian Idahl, Jörg Tiedemann, Sampo Pyysalo +19
Open web-scale pre-training corpora remain concentrated in English, limiting multilingual LLM development. We introduce MultiSynt/MT, an open synthetic parallel corpus with approxi…
SWEb: A Large Web Dataset for the Scandinavian Languages
Tobias Norlund, Tim Isbister, Amaru Cuba Gyllensten +4
This paper presents the hitherto largest pretraining dataset for the Scandinavian languages: the Scandinavian WEb (SWEb), comprising over one trillion tokens. The paper details the…
GPT-SW3: An Autoregressive Language Model for the Nordic Languages
Ariel Ekgren, Amaru Cuba Gyllensten, Felix Stollenwerk +7
This paper details the process of developing the first native large generative language model for the Nordic languages, GPT-SW3. We cover all parts of the development process, from…
The Nordic Pile: A 1.2TB Nordic Dataset for Language Modeling
Joey Öhman, Severine Verlinden, Ariel Ekgren +5
Pre-training Large Language Models (LLMs) require massive amounts of text data, and the performance of the LLMs typically correlates with the scale and quality of the datasets. Thi…
Cross-lingual Transfer of Monolingual Models
Evangelia Gogoulou, Ariel Ekgren, Tim Isbister +1
Recent studies in zero-shot cross-lingual learning using multilingual models have falsified the previous hypothesis that shared vocabulary and joint pre-training are the keys to cr…
Should we Stop Training More Monolingual Models, and Simply Use Machine Translation Instead?
Tim Isbister, Fredrik Carlsson, Magnus Sahlgren
Most work in NLP makes the assumption that it is desirable to develop solutions in the native language in question. There is consequently a strong trend towards building native lan…
Why Not Simply Translate? A First Swedish Evaluation Benchmark for Semantic Similarity
Tim Isbister, Magnus Sahlgren
This paper presents the first Swedish evaluation benchmark for textual semantic similarity. The benchmark is compiled by simply running the English STS-B dataset through the Google…
Automatic Extraction of Personality from Text: Challenges and Opportunities
Nazar Akrami, Johan Fernquist, Tim Isbister +2
In this study, we examined the possibility to extract personality traits from a text. We created an extensive dataset by having experts annotate personality traits in a large numbe…
Monitoring Targeted Hate in Online Environments
Tim Isbister, Magnus Sahlgren, Lisa Kaati +2
Hateful comments, swearwords and sometimes even death threats are becoming a reality for many people today in online environments. This is especially true for journalists, politici…