118 citations · 156 across the 8 of their papers we have counts for
9 papers
Unfair and Unsafe: Everyday Bias Triggers Emergent Misalignment
Alex Martí Guiu, Nicolas Zumarraga, Kevin O'Sullivan +2
Emergent misalignment (EM) occurs when fine-tuning a language model to a narrow task makes it misaligned on unrelated ones. Because organizations now routinely fine-tune models on…
Building Fast, Evaluating Slow: Pipeline Choices Dominate Autointerpretability Score Variance
Sinie van der Ben, Neele Roch, Anna Hedström +1
Cross-paper comparison of sparse autoencoder (SAE) interpretability often relies on autointerpretability scores. In this evaluation pipeline, a language model (LM) explains each fe…
CoSy: Evaluating Textual Explanations of Neurons
Laura Kopf, Philine Lou Bommer, Anna Hedström +3
A crucial aspect of understanding the complex nature of Deep Neural Networks (DNNs) is the ability to explain learned concepts within their latent representations. While methods ex…
Sanity Checks Revisited: An Exploration to Repair the Model Parameter Randomisation Test
Anna Hedström, Leander Weber, Sebastian Lapuschkin +1
The Model Parameter Randomisation Test (MPRT) is widely acknowledged in the eXplainable Artificial Intelligence (XAI) community for its well-motivated evaluative principle: that th…
Explainable AI in Grassland Monitoring: Enhancing Model Performance and Domain Adaptability
Shanghua Liu, Anna Hedström, Deepak Hanike Basavegowda +2
Grasslands are known for their high biodiversity and ability to provide multiple ecosystem services. Challenges in automating the identification of indicator plants are key obstacl…
Finding the right XAI method -- A Guide for the Evaluation and Ranking of Explainable AI Methods in Climate Science
Philine Bommer, Marlene Kretschmer, Anna Hedström +2
Explainable artificial intelligence (XAI) methods shed light on the predictions of machine learning algorithms. Several different approaches exist and have already been applied in…
The Meta-Evaluation Problem in Explainable AI: Identifying Reliable Estimators with MetaQuantus
Anna Hedström, Philine Bommer, Kristoffer K. Wickstrøm +3
One of the unsolved challenges in the field of Explainable AI (XAI) is determining how to most reliably estimate the quality of an explanation method in the absence of ground truth…
Quantus: An Explainable AI Toolkit for Responsible Evaluation of Neural Network Explanations and Beyond
Anna Hedström, Leander Weber, Dilyara Bareeva +5
The evaluation of explanation methods is a research topic that has not yet been explored deeply, however, since explainability is supposed to strengthen trust in artificial intelli…
NoiseGrad: Enhancing Explanations by Introducing Stochasticity to Model Weights
Kirill Bykov, Anna Hedström, Shinichi Nakajima +1
Many efforts have been made for revealing the decision-making process of black-box learning machines such as deep neural networks, resulting in useful local and global explanation…