258 citations · 1.2k across the 64 of their papers we have counts for
27 papers · 1 filter
Efficiently Approximating Attention Is Hard
Lukas Haverbeck, Carmen Amo Alonso, Andres Felipe Posada-Moreno +2
Softmax attention is ubiquitous in modern machine learning, but its quadratic scaling with sequence length makes it costly. To reduce this cost, attention is often approximated wit…
CENDRe: Concept Extraction with Natural Domain Representations
Antonia Holzapfel, Andres Felipe Posada Moreno, Sebastian Trimpe
Convolutional neural networks (CNNs) are widely used for time-series classification, but their deployment in critical domains requires understanding the temporal and spectral patte…
The risk of KV cache compression
Lukas Haverbeck, Carmen Amo Alonso, Andres Felipe Posada-Moreno +2
Transformer inference on long sequences is expensive because softmax attention repeatedly reads from a large KV cache. The prevalent approach to this bottleneck is KV cache compres…
Going Beyond the Edge: Distributed Inference of Transformer Models on Ultra-Low-Power Wireless Devices
Alexander Gräfe, Ding Huo, Vincent de Bakker +3
Transformer models are rapidly becoming a cornerstone of modern Internet of Things (IoT) applications, yet their computational and memory demands far exceed the capabilities of a s…
Viability of Future Actions: Robust Safety in Reinforcement Learning via Entropy Regularization
Pierre-François Massiani, Alexander von Rohr, Lukas Haverbeck +1
Despite the many recent advances in reinforcement learning (RL), the question of learning policies that robustly satisfy state constraints under unknown disturbances remains open.…
Kernel conditional tests from learning-theoretic bounds
Pierre-François Massiani, Christian Fiedler, Lukas Haverbeck +2
We propose a framework for hypothesis testing on conditional probability distributions, which we then use to construct statistical tests of functionals of conditional distributions…
Concept Extraction for Time Series with ECLAD-ts
Antonia Holzapfel, Andres Felipe Posada-Moreno, Sebastian Trimpe
Convolutional neural networks (CNNs) for time series classification (TSC) are being increasingly used in applications ranging from quality prediction to medical diagnosis. The blac…
On Rollouts in Model-Based Reinforcement Learning
Bernd Frauenknecht, Devdutt Subhasish, Friedrich Solowjow +1
Model-based reinforcement learning (MBRL) seeks to enhance data efficiency by learning a model of the environment and generating synthetic rollouts from it. However, accumulated mo…
On Foundation Models for Dynamical Systems from Purely Synthetic Data
Martin Ziegler, Andres Felipe Posada-Moreno, Friedrich Solowjow +1
Foundation models have demonstrated remarkable generalization, data efficiency, and robustness properties across various domains. In this paper, we explore the feasibility of found…
Bayesian Optimization via Continual Variational Last Layer Training
Paul Brunzema, Mikkel Jordahn, John Willes +3
Gaussian Processes (GPs) are widely seen as the state-of-the-art surrogate models for Bayesian optimization (BO) due to their ability to model uncertainty and their performance on…
Trust the Model Where It Trusts Itself -- Model-Based Actor-Critic with Uncertainty-Aware Rollout Adaption
Bernd Frauenknecht, Artur Eisele, Devdutt Subhasish +2
Dyna-style model-based reinforcement learning (MBRL) combines model-free agents with predictive transition models through model-based rollouts. This combination raises a critical q…
On the Consistency of Kernel Methods with Dependent Observations
Pierre-François Massiani, Sebastian Trimpe, Friedrich Solowjow
The consistency of a learning method is usually established under the assumption that the observations are a realization of an independent and identically distributed (i.i.d.) or m…
Learning Hybrid Dynamics Models With Simulator-Informed Latent States
Katharina Ensinger, Sebastian Ziesche, Sebastian Trimpe
Dynamics model learning deals with the task of inferring unknown dynamics from measurement data and predicting the future behavior of the system. A typical approach to address this…
Exact Inference for Continuous-Time Gaussian Process Dynamics
Katharina Ensinger, Nicholas Tagliapietra, Sebastian Ziesche +1
Physical systems can often be described via a continuous-time dynamical system. In practice, the true system is often unknown and has to be learned from measurement data. Since dat…
Multimodal Multi-User Surface Recognition with the Kernel Two-Sample Test
Behnam Khojasteh, Friedrich Solowjow, Sebastian Trimpe +1
Machine learning and deep learning have been used extensively to classify physical surfaces through images and time-series contact data. However, these methods rely on human expert…
Combining Slow and Fast: Complementary Filtering for Dynamics Learning
Katharina Ensinger, Sebastian Ziesche, Barbara Rakitsch +2
Modeling an unknown dynamical system is crucial in order to predict the future behavior of the system. A standard approach is training recurrent models on measurement data. While t…
Event-Triggered Time-Varying Bayesian Optimization
Paul Brunzema, Alexander von Rohr, Friedrich Solowjow +1
We consider the problem of sequentially optimizing a time-varying objective function using time-varying Bayesian optimization (TVBO). Current approaches to TVBO require prior knowl…
On Controller Tuning with Time-Varying Bayesian Optimization
Paul Brunzema, Alexander von Rohr, Sebastian Trimpe
Changing conditions or environments can cause system dynamics to vary over time. To ensure optimal control performance, controllers should adapt to these changes. When the underlyi…
Learning by Doing: Controlling a Dynamical System using Causality, Control, and Reinforcement Learning
Sebastian Weichwald, Søren Wengel Mogensen, Tabitha Edith Lee +6
Questions in causality, control, and reinforcement learning go beyond the classical machine learning task of prediction under i.i.d. observations. Instead, these fields consider th…
GoSafeOpt: Scalable Safe Exploration for Global Optimization of Dynamical Systems
Bhavya Sukhija, Matteo Turchetta, David Lindner +3
Learning optimal control policies directly on physical systems is challenging since even a single failure can lead to costly hardware damage. Most existing model-free learning meth…
Local policy search with Bayesian optimization
Sarah Müller, Alexander von Rohr, Sebastian Trimpe
Reinforcement learning (RL) aims to find an optimal policy by interaction with an environment. Consequently, learning complex behavior requires a vast number of samples, which can…
Structure-preserving Gaussian Process Dynamics
Katharina Ensinger, Friedrich Solowjow, Sebastian Ziesche +2
Most physical processes posses structural properties such as constant energies, volumes, and other invariants over time. When learning models of such dynamical systems, it is criti…
Identifying Causal Structure in Dynamical Systems
Dominik Baumann, Friedrich Solowjow, Karl H. Johansson +1
Mathematical models are fundamental building blocks in the design of dynamical control systems. As control systems are becoming increasingly complex and networked, approaches for o…
Excursion Search for Constrained Bayesian Optimization under a Limited Budget of Failures
Alonso Marco, Alexander von Rohr, Dominik Baumann +2
When learning to ride a bike, a child falls down a number of times before achieving the first success. As falling down usually has only mild consequences, it can be seen as a toler…