Interpreting Face Inference Models using Hierarchical Network Dissection
arXiv:2108.10360 · doi:10.1007/s11263-022-01603-x
Abstract
This paper presents Hierarchical Network Dissection, a general pipeline to interpret the internal representation of face-centric inference models. Using a probabilistic formulation, our pipeline pairs units of the model with concepts in our "Face Dictionary", a collection of facial concepts with corresponding sample images. Our pipeline is inspired by Network Dissection, a popular interpretability model for object-centric and scene-centric models. However, our formulation allows to deal with two important challenges of face-centric models that Network Dissection cannot address: (1) spacial overlap of concepts: there are different facial concepts that simultaneously occur in the same region of the image, like "nose" (facial part) and "pointy nose" (facial attribute); and (2) global concepts: there are units with affinity to concepts that do not refer to specific locations of the face (e.g. apparent age). We use Hierarchical Network Dissection to dissect different face-centric inference models trained on widely-used facial datasets. The results show models trained for different tasks learned different internal representations. Furthermore, the interpretability results can reveal some biases in the training data and some interesting characteristics of the face-centric inference tasks. Finally, we conduct controlled experiments on biased data to showcase the potential of Hierarchical Network Dissection for bias discovery. The results illustrate how Hierarchical Network Dissection can be used to discover and quantify bias in the training data that is also encoded in the model.
References in corpus (7)
- FaceNet: A Unified Embedding for Face Recognition and Clustering
- Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps
- Understanding the Role of Individual Units in a Deep Neural Network
- Synthesizing the preferred inputs for neurons in neural networks via deep generator networks
- GAN Dissection: Visualizing and Understanding Generative Adversarial Networks
- FairFace: Face Attribute Dataset for Balanced Race, Gender, and Age
- Learning De-biased Representations with Biased Representations