13 citations · 15 across the 5 of their papers we have counts for
9 papers
Efficiently Approximating Attention Is Hard
Lukas Haverbeck, Carmen Amo Alonso, Andres Felipe Posada-Moreno +2
Softmax attention is ubiquitous in modern machine learning, but its quadratic scaling with sequence length makes it costly. To reduce this cost, attention is often approximated wit…
The risk of KV cache compression
Lukas Haverbeck, Carmen Amo Alonso, Andres Felipe Posada-Moreno +2
Transformer inference on long sequences is expensive because softmax attention repeatedly reads from a large KV cache. The prevalent approach to this bottleneck is KV cache compres…
Sailing Towards Zero-Shot State Estimation using Foundation Models Combined with a UKF
Tobin Holtmann, David Stenger, Andres Posada-Moreno +2
State estimation in control and systems engineering traditionally requires extensive manual system identification or data-collection effort. However, transformer-based foundation m…
Concept Extraction for Time Series with ECLAD-ts
Antonia Holzapfel, Andres Felipe Posada-Moreno, Sebastian Trimpe
Convolutional neural networks (CNNs) for time series classification (TSC) are being increasingly used in applications ranging from quality prediction to medical diagnosis. The blac…
On Foundation Models for Dynamical Systems from Purely Synthetic Data
Martin Ziegler, Andres Felipe Posada-Moreno, Friedrich Solowjow +1
Foundation models have demonstrated remarkable generalization, data efficiency, and robustness properties across various domains. In this paper, we explore the feasibility of found…
Scale-Preserving Automatic Concept Extraction (SPACE)
Andrés Felipe Posada-Moreno, Lukas Kreisköther, Tassilo Glander +1
Convolutional Neural Networks (CNN) have become a common choice for industrial quality control, as well as other critical applications in the Industry 4.0. When these CNNs behave i…
Scalable Concept Extraction in Industry 4.0
Andrés Felipe Posada-Moreno, Kai Müller, Florian Brillowski +3
The industry 4.0 is leveraging digital technologies and machine learning techniques to connect and optimize manufacturing processes. Central to this idea is the ability to transfor…
ECLAD: Extracting Concepts with Local Aggregated Descriptors
Andres Felipe Posada-Moreno, Nikita Surya, Sebastian Trimpe
Convolutional neural networks (CNNs) are increasingly being used in critical systems, where robustness and alignment are crucial. In this context, the field of explainable artifici…
Under the Hood of Neural Networks: Characterizing Learned Representations by Functional Neuron Populations and Network Ablations
Richard Meyes, Constantin Waubert de Puiseau, Andres Posada-Moreno +1
The need for more transparency of the decision-making processes in artificial neural networks steadily increases driven by their applications in safety critical and ethically chall…