649 citations · 689 across the 8 of their papers we have counts for
8 papers
Surrogate Fidelity: When Can Open LLMs Explain Closed Ones?
Philippe Chlenski, Zachariah Carmichael, Ayush Warikoo +5
Mechanistic interpretability (MI) requires full access to model internals, yet the APIs for most widely deployed language models at best expose log-probabilities over output tokens…
On Symmetries in Convolutional Weights
Bilal Alsallakh, Timothy Wroge, Vivek Miglani +1
We explore the symmetry of the mean k x k weight kernel in each layer of various convolutional neural networks. Unlike individual neurons, the mean kernels in internal layers tend…
Using Captum to Explain Generative Language Models
Vivek Miglani, Aobo Yang, Aram H. Markosyan +2
Captum is a comprehensive library for model explainability in PyTorch, offering a range of methods from the interpretability literature to enhance users' understanding of PyTorch m…
Bias Mitigation Framework for Intersectional Subgroups in Neural Networks
Narine Kokhlikyan, Bilal Alsallakh, Fulton Wang +3
We propose a fairness-aware learning framework that mitigates intersectional subgroup bias associated with protected attributes. Prior research has primarily focused on mitigating…
Investigating sanity checks for saliency maps with image and text classification
Narine Kokhlikyan, Vivek Miglani, Bilal Alsallakh +2
Saliency maps have shown to be both useful and misleading for explaining model predictions especially in the context of images. In this paper, we perform sanity checks for text mod…
Investigating Saturation Effects in Integrated Gradients
Vivek Miglani, Narine Kokhlikyan, Bilal Alsallakh +2
Integrated Gradients has become a popular method for post-hoc model interpretability. De-spite its popularity, the composition and relative impact of different regions of the integ…
Mind the Pad -- CNNs can Develop Blind Spots
Bilal Alsallakh, Narine Kokhlikyan, Vivek Miglani +2
We show how feature maps in convolutional networks are susceptible to spatial bias. Due to a combination of architectural choices, the activation at certain locations is systematic…
Captum: A unified and generic model interpretability library for PyTorch
Narine Kokhlikyan, Vivek Miglani, Miguel Martin +8
In this paper we introduce a novel, unified, open-source model interpretability library for PyTorch [12]. The library contains generic implementations of a number of gradient and p…