91 citations · 207 across the 22 of their papers we have counts for
24 papers
Hyperband-based Bayesian Optimization for Black-box Prompt Selection
Lennart Schneider, Martin Wistuba, Aaron Klein +3
Optimal prompt selection is crucial for maximizing large language model (LLM) performance on downstream tasks, especially in black-box settings where models are only accessible via…
Choice of PEFT Technique in Continual Learning: Prompt Tuning is Not All You Need
Martin Wistuba, Prabhu Teja Sivaprasad, Lukas Balles +1
Recent Continual Learning (CL) methods have combined pretrained Transformers with prompt tuning, a parameter-efficient fine-tuning (PEFT) technique. We argue that the choice of pro…
A Negative Result on Gradient Matching for Selective Backprop
Lukas Balles, Cedric Archambeau, Giovanni Zappella
With increasing scale in model and dataset size, the training of deep neural networks becomes a massive computational burden. One approach to speed up the training process is Selec…
Continual Learning with Low Rank Adaptation
Martin Wistuba, Prabhu Teja Sivaprasad, Lukas Balles +1
Recent work using pretrained transformers has shown impressive performance when fine-tuned with data from the downstream problem of interest. However, they struggle to retain that…
Renate: A Library for Real-World Continual Learning
Martin Wistuba, Martin Ferianc, Lukas Balles +2
Continual learning enables the incremental training of machine learning models on non-stationary data streams.While academic interest in the topic is high, there is little indicati…
PASHA: Efficient HPO and NAS with Progressive Resource Allocation
Ondrej Bohdal, Lukas Balles, Martin Wistuba +3
Hyperparameter optimization (HPO) and neural architecture search (NAS) are methods of choice to obtain the best-in-class machine learning models, but in practice they can be costly…