3 papers
cs.CL2026
Is a Document Educational or Just Wikipedia-Style? -- Pitfalls of Classifier-Based Quality Filtering
Mateusz Klimaszewski, Piotr Andruszkiewicz
Classifier-based Quality Filtering has recently emerged as a fundamental technique in constructing pre-training corpora. The ability to deploy a single model that can replace or su…
cs.CL2024
No Train but Gain: Language Arithmetic for training-free Language Adapters enhancement
Mateusz Klimaszewski, Piotr Andruszkiewicz, Alexandra Birch
Modular deep learning is the state-of-the-art solution for lifting the curse of multilinguality, preventing the impact of negative interference and enabling cross-lingual performan…
cs.CL2024
Is Modularity Transferable? A Case Study through the Lens of Knowledge Distillation
Mateusz Klimaszewski, Piotr Andruszkiewicz, Alexandra Birch
The rise of Modular Deep Learning showcases its potential in various Natural Language Processing applications. Parameter-efficient fine-tuning (PEFT) modularity has been shown to w…