25.8k citations
- Central European Institute of TechnologyCZ145 papers
- Centre National de la Recherche ScientifiqueFR42 papers
- Masaryk UniversityCZ41 papers
- TU WienAT34 papers
- Université Grenoble AlpesFR24 papers
- FZU ‒ Institute of Physics of the Academy of Sciences of the Czech RepublicCZ22 papers
- University of ViennaAT21 papers
- Charles UniversityCZ17 papers
- Central European Institute of Technology – Masaryk UniversityCZ16 papers
- Kempelen Institute of Intelligent Technologies13 papers
- Laboratoire National des Champs Magnétiques IntensesFR13 papers
- Military University of Technology in WarsawPL13 papers
30 papers · 1 filter
PEFT-Factory: Unified Parameter-Efficient Fine-Tuning of Autoregressive Large Language Models
Robert Belanec, Ivan Srba, Maria Bielikova
Parameter-Efficient Fine-Tuning (PEFT) methods address the increasing size of Large Language Models (LLMs). Currently, many newly introduced PEFT methods are challenging to replica…
PEFT-Bench: A Parameter-Efficient Fine-Tuning Methods Benchmark
Robert Belanec, Branislav Pecher, Ivan Srba +1
Despite the state-of-the-art performance of Large Language Models (LLMs) achieved on many tasks, their massive scale often leads to high computational and environmental costs, limi…
Task Prompt Vectors: Effective Initialization through Multi-Task Soft-Prompt Transfer
Robert Belanec, Simon Ostermann, Ivan Srba +1
Prompt tuning is an efficient solution for training large language models (LLMs). However, current soft-prompt-based methods often sacrifice multi-task modularity, requiring the tr…
Soft Language Prompts for Language Transfer
Ivan Vykopal, Simon Ostermann, Marián Šimko
Cross-lingual knowledge transfer, especially between high- and low-resource languages, remains challenging in natural language processing (NLP). This study offers insights for impr…
Fighting Randomness with Randomness: Mitigating Optimisation Instability of Fine-Tuning using Delayed Ensemble and Noisy Interpolation
Branislav Pecher, Jan Cegin, Robert Belanec +3
While fine-tuning of pre-trained language models generally helps to overcome the lack of labelled training samples, it also displays model performance instability. This instability…
DAIC-WOZ: On the Validity of Using the Therapist's prompts in Automatic Depression Detection from Clinical Interviews
Sergio Burdisso, Ernesto Reyes-Ramírez, Esaú Villatoro-Tello +3
Automatic depression detection from conversational data has gained significant interest in recent years. The DAIC-WOZ dataset, interviews conducted by a human-controlled virtual ag…