3 citations · 8 across the 10 of their papers we have counts for
3 papers · 1 filter
The ParlaSpeech Collection of Automatically Generated Speech and Text Datasets from Parliamentary Proceedings
Nikola Ljubešić, Peter Rupnik, Danijel Koržinek
Recent significant improvements in speech and language technologies come both from self-supervised approaches over raw language data as well as various types of explicit supervisio…
Language Models on a Diet: Cost-Efficient Development of Encoders for Closely-Related Languages via Additional Pretraining
Nikola Ljubešić, Vít Suchomel, Peter Rupnik +2
The world of language models is going through turbulent times, better and ever larger models are coming out at an unprecedented speed. However, we argue that, especially for the sc…
Do Language Models Care About Text Quality? Evaluating Web-Crawled Corpora Across 11 Languages
Rik van Noord, Taja Kuzman, Peter Rupnik +4
Large, curated, web-crawled corpora play a vital role in training language models (LMs). They form the lion's share of the training data in virtually all recent LMs, such as the we…