5 papers
Environment-Grounded Automated Prompt Optimization for LLM Game Agents
Rean Clive Fernandes, Lukas Fehring, Theresa Eimer +2
LLM agents in interactive environments are highly sensitive to their prompts, yet prompt engineering remains a manual, task-specific process. We introduce an automated prompt optim…
Provably Reduced Sample Cost in Prior-Guided Hyperparameter Optimization
Leona Hennig, Jasmin Brandt, Lukas Fehring +3
Large-scale hyperparameter optimization (HPO) in automated machine learning (AutoML) consumes substantial computational resources, raising growing concerns about scalability and en…
Neural Attention Search Linear: Towards Adaptive Token-Level Hybrid Attention Models
Difan Deng, Andreas Bentzen Winje, Lukas Fehring +1
The quadratic computational complexity of softmax transformers has become a bottleneck in long-context scenarios. In contrast, linear attention model families provide a promising d…
Dynamic Priors in Bayesian Optimization for Hyperparameter Optimization
Lukas Fehring, Marcel Wever, Maximilian Spliethöver +3
Bayesian optimization (BO) is a widely used approach to hyperparameter optimization (HPO). However, most existing HPO methods only incorporate expert knowledge during initializatio…
Growing with Experience: Growing Neural Networks in Deep Reinforcement Learning
Lukas Fehring, Marius Lindauer, Theresa Eimer
While increasingly large models have revolutionized much of the machine learning landscape, training even mid-sized networks for Reinforcement Learning (RL) is still proving to be…