208 citations · 504 across the 30 of their papers we have counts for
24 papers · 1 filter
Hyperband-based Bayesian Optimization for Black-box Prompt Selection
Lennart Schneider, Martin Wistuba, Aaron Klein +3
Optimal prompt selection is crucial for maximizing large language model (LLM) performance on downstream tasks, especially in black-box settings where models are only accessible via…
Choice of PEFT Technique in Continual Learning: Prompt Tuning is Not All You Need
Martin Wistuba, Prabhu Teja Sivaprasad, Lukas Balles +1
Recent Continual Learning (CL) methods have combined pretrained Transformers with prompt tuning, a parameter-efficient fine-tuning (PEFT) technique. We argue that the choice of pro…
Continual Learning with Low Rank Adaptation
Martin Wistuba, Prabhu Teja Sivaprasad, Lukas Balles +1
Recent work using pretrained transformers has shown impressive performance when fine-tuned with data from the downstream problem of interest. However, they struggle to retain that…
Renate: A Library for Real-World Continual Learning
Martin Wistuba, Martin Ferianc, Lukas Balles +2
Continual learning enables the incremental training of machine learning models on non-stationary data streams.While academic interest in the topic is high, there is little indicati…
Variational Boosted Soft Trees
Tristan Cinquin, Tammo Rukat, Philipp Schmidt +2
Gradient boosting machines (GBMs) based on decision trees consistently demonstrate state-of-the-art results on regression and classification tasks with tabular data, often outperfo…
Scaling Laws for Hyperparameter Optimization
Arlind Kadra, Maciej Janowski, Martin Wistuba +1
Hyperparameter optimization is an important subfield of machine learning that focuses on tuning the hyperparameters of a chosen algorithm to achieve peak performance. Recently, the…
PASHA: Efficient HPO and NAS with Progressive Resource Allocation
Ondrej Bohdal, Lukas Balles, Martin Wistuba +3
Hyperparameter optimization (HPO) and neural architecture search (NAS) are methods of choice to obtain the best-in-class machine learning models, but in practice they can be costly…
Continual Learning with Transformers for Image Classification
Beyza Ermis, Giovanni Zappella, Martin Wistuba +2
In many real-world scenarios, data to train machine learning models become available over time. However, neural network models struggle to continually learn new concepts without fo…
Supervising the Multi-Fidelity Race of Hyperparameter Configurations
Martin Wistuba, Arlind Kadra, Josif Grabocka
Multi-fidelity (gray-box) hyperparameter optimization techniques (HPO) have recently emerged as a promising direction for tuning Deep Learning methods. However, existing methods su…
HPO-B: A Large-Scale Reproducible Benchmark for Black-Box HPO based on OpenML
Sebastian Pineda Arango, Hadi S. Jomaa, Martin Wistuba +1
Hyperparameter optimization (HPO) is a core problem for the machine learning community and remains largely unsolved due to the significant computational resources required to evalu…
A Comprehensive Survey on Hardware-Aware Neural Architecture Search
Hadjer Benmeziane, Kaoutar El Maghraoui, Hamza Ouarnoughi +3
Neural Architecture Search (NAS) methods have been growing in popularity. These techniques have been fundamental to automate and speed up the time consuming and error-prone process…
Few-Shot Bayesian Optimization with Deep Kernel Surrogates
Martin Wistuba, Josif Grabocka
Hyperparameter optimization (HPO) is a central pillar in the automation of machine learning solutions and is mainly performed via Bayesian optimization, where a parametric surrogat…
Learning to Rank Learning Curves
Martin Wistuba, Tejaswini Pedapati
Many automated machine learning methods, such as those for hyperparameter and neural architecture optimization, are computationally expensive because they involve training many dif…
XferNAS: Transfer Neural Architecture Search
Martin Wistuba
The term Neural Architecture Search (NAS) refers to the automatic optimization of network architectures for a new, previously unknown task. Since testing an architecture is computa…
A Survey on Neural Architecture Search
Martin Wistuba, Ambrish Rawat, Tejaswini Pedapati
The growing interest in both the automation of machine learning and deep learning has inevitably led to the development of a wide variety of automated methods for neural architectu…
Optimal Exploitation of Clustering and History Information in Multi-Armed Bandit
Djallel Bouneffouf, Srinivasan Parthasarathy, Horst Samulowitz +1
We consider the stochastic multi-armed bandit problem and the contextual bandit problem with historical observations and pre-clustered arms. The historical observations can contain…
Inductive Transfer for Neural Architecture Optimization
Martin Wistuba, Tejaswini Pedapati
The recent advent of automated neural network architecture search led to several methods that outperform state-of-the-art human-designed architectures. However, these approaches ar…
NeuNetS: An Automated Synthesis Engine for Neural Network Design
Atin Sood, Benjamin Elder, Benjamin Herta +17
Application of neural networks to a vast variety of practical applications is transforming the way AI is applied in practice. Pre-trained neural network models available through AP…
Adversarial Robustness Toolbox v1.0.0
Maria-Irina Nicolae, Mathieu Sinn, Minh Ngoc Tran +9
Adversarial Robustness Toolbox (ART) is a Python library supporting developers and researchers in defending Machine Learning models (Deep Neural Networks, Gradient Boosted Decision…
Scalable Multi-Class Bayesian Support Vector Machines for Structured and Unstructured Data
Martin Wistuba, Ambrish Rawat
We introduce a new Bayesian multi-class support vector machine by formulating a pseudo-likelihood for a multi-class hinge loss in the form of a location-scale mixture of Gaussians.…
Finding Competitive Network Architectures Within a Day Using UCT
Martin Wistuba
The design of neural network architectures for a new data set is a laborious task which requires human deep learning expertise. In order to make deep learning available for a broad…
Bank Card Usage Prediction Exploiting Geolocation Information
Martin Wistuba, Nghia Duong-Trung, Nicolas Schilling +1
We describe the solution of team ISMLL for the ECML-PKDD 2016 Discovery Challenge on Bank Card Usage for both tasks. Our solution is based on three pillars. Gradient boosted decisi…
Ultra-Fast Shapelets for Time Series Classification
Martin Wistuba, Josif Grabocka, Lars Schmidt-Thieme
Time series shapelets are discriminative subsequences and their similarity to a time series can be used for time series classification. Since the discovery of time series shapelets…
Scalable Discovery of Time-Series Shapelets
Josif Grabocka, Martin Wistuba, Lars Schmidt-Thieme
Time-series classification is an important problem for the data mining community due to the wide range of application domains involving time-series data. A recent paradigm, called…