76 citations · 132 across the 19 of their papers we have counts for
21 papers
We Need Explanation Cards to Connect Explanation Algorithms to the Real World
Eric Günther, Balázs Szabados, Kristof Meding +3
Algorithmic explanations are intended to help stakeholders understand opaque algorithmic decisions, but in practice, they often fall short. First, the meaning of algorithmic explan…
Validity Threats for Foundation Model Research
Gunnar König, Martin Pawelczyk, Ulrike von Luxburg +1
Controlled experiments are the backbone of machine learning research, but at the scale of modern foundation models, they have become prohibitively expensive. Instead, the community…
How to Scale Mixture-of-Experts: From muP to the Maximally Scale-Stable Parameterization
Leena Chennuru Vankadara, Moritz Haas, Luke Hayward +2
Recent frontier large language models predominantly rely on Mixture-of-Experts (MoE) architectures. Despite empirical progress, there is still no principled understanding of how hy…
Using predictive multiplicity to measure individual performance within the AI Act
Karolin Frohnapfel, Mara Seyfert, Sebastian Bordt +2
When building AI systems for decision support, one often encounters the phenomenon of predictive multiplicity: a single best model does not exist; instead, one can construct many m…
Weight Decay Improves Language Model Plasticity
Tessa Han, Sebastian Bordt, Hanlin Zhang +1
Large language models are typically trained in two broad phases: pretraining to produce a base model, followed by further training to improve downstream performance. However, hyper…
Train Once, Answer All: Many Pretraining Experiments for the Cost of One
Sebastian Bordt, Martin Pawelczyk
Recent work has demonstrated that controlled pretraining experiments are a powerful tool for studying the relationship between training data and large language model (LLM) behavior…
Informative Post-Hoc Explanations Only Exist for Simple Functions
Eric Günther, Balázs Szabados, Robi Bhattacharjee +2
Many researchers have suggested that local post-hoc explanation algorithms can be used to gain insights into the behavior of complex machine learning models. However, theoretical g…
On the Surprising Effectiveness of Large Learning Rates under Standard Width Scaling
Moritz Haas, Sebastian Bordt, Ulrike von Luxburg +1
Scaling limits, such as infinite-width limits, serve as promising theoretical tools to study large-scale models. However, it is widely believed that existing infinite-width theory…
How Much Can We Forget about Data Contamination?
Sebastian Bordt, Suraj Srinivas, Valentyn Boreiko +1
The leakage of benchmark data into the training data has emerged as a significant challenge for evaluating the capabilities of large language models (LLMs). In this work, we challe…
Elephants Never Forget: Memorization and Learning of Tabular Data in Large Language Models
Sebastian Bordt, Harsha Nori, Vanessa Rodrigues +2
While many have shown how Large Language Models (LLMs) can be applied to a diverse set of tasks, the critical issues of data contamination and memorization are often glossed over.…
Elephants Never Forget: Testing Language Models for Memorization of Tabular Data
Sebastian Bordt, Harsha Nori, Rich Caruana
While many have shown how Large Language Models (LLMs) can be applied to a diverse set of tasks, the critical issues of data contamination and memorization are often glossed over.…
Data Science with LLMs and Interpretable Models
Sebastian Bordt, Ben Lengerich, Harsha Nori +1
Recent years have seen important advances in the building of interpretable models, machine learning models that are designed to be easily understood by humans. In this work, we sho…
Rethinking Explainable Machine Learning as Applied Statistics
Sebastian Bordt, Eric Raidl, Ulrike von Luxburg
In the rapidly growing literature on explanation algorithms, it often remains unclear what precisely these algorithms are for and how they should be used. In this position paper, w…
LLMs Understand Glass-Box Models, Discover Surprises, and Suggest Repairs
Benjamin J. Lengerich, Sebastian Bordt, Harsha Nori +4
We show that large language models (LLMs) are remarkably good at working with interpretable models that decompose complex outcomes into univariate graph-represented components. By…
Which Models have Perceptually-Aligned Gradients? An Explanation via Off-Manifold Robustness
Suraj Srinivas, Sebastian Bordt, Hima Lakkaraju
One of the remarkable properties of robust computer vision models is that their input-gradients are often aligned with human perception, referred to in the literature as perceptual…
ChatGPT Participates in a Computer Science Exam
Sebastian Bordt, Ulrike von Luxburg
We asked ChatGPT to participate in an undergraduate computer science exam on ''Algorithms and Data Structures''. The program was evaluated on the entire exam as posed to the studen…
From Shapley Values to Generalized Additive Models and back
Sebastian Bordt, Ulrike von Luxburg
In explainable machine learning, local post-hoc explanation algorithms and inherently interpretable models are often seen as competing approaches. This work offers a partial reconc…
The Manifold Hypothesis for Gradient-Based Explanations
Sebastian Bordt, Uddeshya Upadhyay, Zeynep Akata +1
When do gradient-based explanation algorithms provide perceptually-aligned explanations? We propose a criterion: the feature attributions need to be aligned with the tangent space…
Post-Hoc Explanations Fail to Achieve their Purpose in Adversarial Contexts
Sebastian Bordt, Michèle Finck, Eric Raidl +1
Existing and planned legislation stipulates various obligations to provide information about machine learning algorithms and their functioning, often interpreted as obligations to…
Recovery Guarantees for Kernel-based Clustering under Non-parametric Mixture Models
Leena Chennuru Vankadara, Sebastian Bordt, Ulrike von Luxburg +1
Despite the ubiquity of kernel-based clustering, surprisingly few statistical guarantees exist beyond settings that consider strong structural assumptions on the data generation pr…
A Bandit Model for Human-Machine Decision Making with Private Information and Opacity
Sebastian Bordt, Ulrike von Luxburg
Applications of machine learning inform human decision makers in a broad range of tasks. The resulting problem is usually formulated in terms of a single decision maker. We argue t…