2 citations · 3 across the 8 of their papers we have counts for
6 papers · 1 filter
How Can We Synthesize High-Quality Pretraining Data? A Systematic Study of Prompt Design, Generator Model, and Source Data
Joel Niklaus, Atsuki Yamaguchi, Michal Štefánik +9
Synthetic data is a standard component in training large language models, yet systematic comparisons across design dimensions, including rephrasing strategy, generator model, and s…
Think Twice: Measuring the Efficiency of Eliminating Prediction Shortcuts of Question Answering Models
Lukáš Mikula, Michal Štefánik, Marek Petrovič +1
While the Large Language Models (LLMs) dominate a majority of language understanding tasks, previous work shows that some of these results are supported by modelling spurious corre…
Resources and Few-shot Learners for In-context Learning in Slavic Languages
Michal Štefánik, Marek Kadlčík, Piotr Gramacki +1
Despite the rapid recent progress in creating accurate and compact in-context learners, most recent work focuses on in-context learning (ICL) for tasks in English. However, the abi…
Soft Alignment Objectives for Robust Adaptation of Language Generation
Michal Štefánik, Marek Kadlčík, Petr Sojka
Domain adaptation allows generative language models to address specific flaws caused by the domain shift of their application. However, the traditional adaptation by further traini…
Adaptor: Objective-Centric Adaptation Framework for Language Models
Michal Štefánik, Vít Novotný, Nikola Groverová +1
Progress in natural language processing research is catalyzed by the possibilities given by the widespread software frameworks. This paper introduces Adaptor library that transpose…
One Size Does Not Fit All: Finding the Optimal Subword Sizes for FastText Models across Languages
Vít Novotný, Eniafe Festus Ayetiran, Dalibor Bačovský +3
Unsupervised representation learning of words from large multilingual corpora is useful for downstream tasks such as word sense disambiguation, semantic text similarity, and inform…