Understanding Instance-based Interpretability of Variational Auto-Encoders

Abstract

Instance-based interpretation methods have been widely studied for supervised learning methods as they help explain how black box neural networks predict. However, instance-based interpretations remain ill-understood in the context of unsupervised learning. In this paper, we investigate influence functions [Koh and Liang, 2017], a popular instance-based interpretation method, for a class of deep generative models called variational auto-encoders (VAE). We formally frame the counter-factual question answered by influence functions in this setting, and through theoretical analysis, examine what they reveal about the impact of training samples on classical unsupervised learning methods. We then introduce VAE- TracIn, a computationally efficient and theoretically sound solution based on Pruthi et al. [2020], for VAEs. Finally, we evaluate VAE-TracIn on several real world datasets with extensive quantitative and qualitative analysis.

NeurIPS 2021

References in corpus (6)

• Data Shapley: Equitable Valuation of Data for Machine Learning
• Disentangling Factors of Variation via Generative Entangling
• Data Cleansing for Models Trained with SGD
• A Non-Parametric Test to Detect Data-Copying in Generative Models
• Out-of-Distribution Generalization Analysis via Influence Function
• Multi-Stage Influence Function

Cited by in corpus (3)

• Rethinking Influence Functions of Neural Networks in the Over-parameterized Regime
• TracInAD: Measuring Influence for Anomaly Detection
• Differential Privacy, Linguistic Fairness, and Training Data Influence: Impossibility and Possibility Theorems for Multilingual Language Models