170 citations · 203 across the 21 of their papers we have counts for
23 papers
ResLRP: The Role of Residual Cancellation in Attribution Instability in Vision Transformers
Jim Berend, Reduan Achtibat, Daniel Schäffer +4
Vision Transformers (ViTs) are central to most modern vision models, yet obtaining input attributions that are fine-grained, faithful, and stable remains challenging. Layer-wise Re…
Safety Signals to Verify NetOps Agents with Action-Level Granularity
Tobias Labarta, Frederik Pahde, Novak Boškov +6
Agentic Network Operations (NetOps) are an emerging paradigm promising to enable workload-aware, self-adjustable, and reliable autonomous networks. While agents have proven their v…
ICON decomposition: Multivariate concept auditing reveals shortcuts that neural networks rely on
Roshan Prakash Rane, Marco Simnacher, Manuel Pfeuffer +7
Deep neural networks often exploit spurious associations, a failure known as shortcut learning. Before deployment, models should therefore be audited for reliance on concepts such…
Concept-based explanation of gene expression prediction from H&E images
Amos Muench, Jonathan Thielmann, Reduan Achtibat +9
Recent advances in pathology foundation models have enabled accurate prediction of spatial transcriptomics (ST) from routine H&E images. However, existing explainability methods fo…
From Attribution to Action: A Human-Centered Application of Activation Steering
Tobias Labarta, Maximilian Dreyer, Katharina Weitz +2
Explainable AI (XAI) methods reveal which features influence model predictions, yet provide limited means for practitioners to act on these explanations. Activation steering of com…
Contrastive Semantic Projection: Faithful Neuron Labeling with Contrastive Examples
Oussama Bouanani, Jim Berend, Wojciech Samek +2
Neuron labeling assigns textual descriptions to internal units of deep networks. Existing approaches typically rely on highly activating examples, often yielding broad or misleadin…
X-SYS: A Reference Architecture for Interactive Explanation Systems
Tobias Labarta, Nhi Hoang, Maximilian Dreyer +5
The explainable AI (XAI) research community has proposed numerous technical methods, yet deploying explainability as systems remains challenging: Interactive explanation systems re…
Dyslexify: A Mechanistic Defense Against Typographic Attacks in CLIP
Lorenz Hufe, Constantin Venhoff, Erblina Purelku +3
Typographic attacks exploit multi-modal systems by injecting text into images, leading to targeted misclassifications, malicious content generation and even Vision-Language Model j…
Attribution-Guided Pruning for Insight and Control: Circuit Discovery and Targeted Correction in Small-scale LLMs
Sayed Mohammad Vakilzadeh Hatefi, Maximilian Dreyer, Reduan Achtibat +5
Large Language Models (LLMs) are widely deployed in real-world applications, yet their internal mechanisms remain difficult to interpret and control, limiting our ability to diagno…
From What to How: Attributing CLIP's Latent Components Reveals Unexpected Semantic Reliance
Maximilian Dreyer, Lorenz Hufe, Jim Berend +3
Transformer-based CLIP models are widely used for text-image probing and feature extraction, making it relevant to understand the internal mechanisms behind their predictions. Whil…
Mechanistic understanding and validation of large AI models with SemanticLens
Maximilian Dreyer, Jim Berend, Tobias Labarta +4
Unlike human-engineered systems such as aeroplanes, where each component's role and dependencies are well understood, the inner workings of AI models remain largely opaque, hinderi…
Pruning By Explaining Revisited: Optimizing Attribution Methods to Prune CNNs and Transformers
Sayed Mohammad Vakilzadeh Hatefi, Maximilian Dreyer, Reduan Achtibat +3
To solve ever more complex problems, Deep Neural Networks are scaled to billions of parameters, leading to huge computational costs. An effective approach to reduce computational r…
Explainable concept mappings of MRI: Revealing the mechanisms underlying deep learning-based brain disease classification
Christian Tinauer, Anna Damulina, Maximilian Sackl +9
Motivation. While recent studies show high accuracy in the classification of Alzheimer's disease using deep neural networks, the underlying learned concepts have not been investiga…
Reactive Model Correction: Mitigating Harm to Task-Relevant Features via Conditional Bias Suppression
Dilyara Bareeva, Maximilian Dreyer, Frederik Pahde +2
Deep Neural Networks are prone to learning and relying on spurious correlations in the training data, which, for high-risk applications, can have fatal consequences. Various approa…
PURE: Turning Polysemantic Neurons Into Pure Features by Identifying Relevant Circuits
Maximilian Dreyer, Erblina Purelku, Johanna Vielhaben +2
The field of mechanistic interpretability aims to study the role of individual neurons in Deep Neural Networks. Single neurons, however, have the capability to act polysemantically…
AttnLRP: Attention-Aware Layer-Wise Relevance Propagation for Transformers
Reduan Achtibat, Sayed Mohammad Vakilzadeh Hatefi, Maximilian Dreyer +4
Large Language Models are prone to biased predictions and hallucinations, underlining the paramount importance of understanding their model-internal reasoning process. However, ach…
Understanding the (Extra-)Ordinary: Validating Deep Model Decisions with Prototypical Concept-based Explanations
Maximilian Dreyer, Reduan Achtibat, Wojciech Samek +1
Ensuring both transparency and safety is critical when deploying Deep Neural Networks (DNNs) in high-risk applications, such as medicine. The field of explainable AI (XAI) has prop…
From Hope to Safety: Unlearning Biases of Deep Models via Gradient Penalization in Latent Space
Maximilian Dreyer, Frederik Pahde, Christopher J. Anders +2
Deep Neural Networks are prone to learning spurious correlations embedded in the training data, leading to potentially biased predictions. This poses risks when deploying these mod…
Reveal to Revise: An Explainable AI Life Cycle for Iterative Bias Correction of Deep Models
Frederik Pahde, Maximilian Dreyer, Wojciech Samek +1
State-of-the-art machine learning models often learn spurious correlations embedded in the training data. This poses risks when deploying these models for high-stake decision-makin…
Revealing Hidden Context Bias in Segmentation and Object Detection through Concept-specific Explanations
Maximilian Dreyer, Reduan Achtibat, Thomas Wiegand +2
Applying traditional post-hoc attribution methods to segmentation or object detection predictors offers only limited insights, as the obtained feature attribution maps at input lev…
From Attribution Maps to Human-Understandable Explanations through Concept Relevance Propagation
Reduan Achtibat, Maximilian Dreyer, Ilona Eisenbraun +4
The field of eXplainable Artificial Intelligence (XAI) aims to bring transparency to today's powerful but opaque deep learning models. While local XAI methods explain individual pr…
Navigating Neural Space: Revisiting Concept Activation Vectors to Overcome Directional Divergence
Frederik Pahde, Maximilian Dreyer, Leander Weber +5
With a growing interest in understanding neural network prediction strategies, Concept Activation Vectors (CAVs) have emerged as a popular tool for modeling human-understandable co…
ECQ: Explainability-Driven Quantization for Low-Bit and Sparse DNNs
Daniel Becking, Maximilian Dreyer, Wojciech Samek +2
The remarkable success of deep neural networks (DNNs) in various applications is accompanied by a significant increase in network parameters and arithmetic operations. Such increas…