3.2k citations · 7.4k across the 37 of their papers we have counts for
37 papers
Don't trust your eyes: on the (un)reliability of feature visualizations
Robert Geirhos, Roland S. Zimmermann, Blair Bilodeau +2
How do neural networks extract patterns from pixels? Feature visualizations attempt to answer this important question by visualizing highly activating patterns through optimization…
Gaussian Process Probes (GPP) for Uncertainty-Aware Probing
Zi Wang, Alexander Ku, Jason Baldridge +2
Understanding which concepts models can and cannot represent has been fundamental to many tasks: from effective and responsible use of models to detecting out of distribution data.…
Does Localization Inform Editing? Surprising Differences in Causality-Based Localization vs. Knowledge Editing in Language Models
Peter Hase, Mohit Bansal, Been Kim +1
Language models learn a great quantity of factual information during pretraining, and recent work localizes this information to specific model weights like mid-layer MLP weights. I…
Post hoc Explanations may be Ineffective for Detecting Unknown Spurious Correlation
Julius Adebayo, Michael Muelly, Hal Abelson +1
We investigate whether three types of post hoc model explanations--feature attribution, concept activation, and training point ranking--are effective for detecting a model's relian…
Impossibility Theorems for Feature Attribution
Blair Bilodeau, Natasha Jaques, Pang Wei Koh +1
Despite a sea of interpretability methods that can produce plausible explanations, the field has also empirically seen many failure cases of such methods. In light of these results…
Human-Centered Concept Explanations for Neural Networks
Chih-Kuan Yeh, Been Kim, Pradeep Ravikumar
Understanding complex machine learning models such as deep neural networks with explanations is crucial in various applications. Many explanations stem from the model perspective,…