Classifying the classifier: dissecting the weight space of neural networks
arXiv:2002.05688
Abstract
This paper presents an empirical study on the weights of neural networks, where we interpret each model as a point in a high-dimensional space -- the neural weight space. To explore the complex structure of this space, we sample from a diverse selection of training variations (dataset, optimization procedure, architecture, etc.) of neural network classifiers, and train a large number of models to represent the weight space. Then, we use a machine learning approach for analyzing and extracting information from this space. Most centrally, we train a number of novel deep meta-classifiers with the objective of classifying different properties of the training setup by identifying their footprints in the weight space. Thus, the meta-classifiers probe for patterns induced by hyper-parameters, so that we can quantify how much, where, and when these are encoded through the optimization process. This provides a novel and complementary view for explainable AI, and we show how meta-classifiers can reveal a great deal of information about the training setup and optimization, by only considering a small subset of randomly selected consecutive weights. To promote further research on the weight space, we release the neural weight space (NWS) dataset -- a collection of 320K weight snapshots from 16K individually trained deep neural networks.
arXiv commentECAI 2020
References in corpus (17)
- Adam: A Method for Stochastic Optimization
- Very Deep Convolutional Networks for Large-Scale Image Recognition
- Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
- UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction
- Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms
- How transferable are features in deep neural networks?
- Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs)
- Comprehensive Privacy Analysis of Deep Learning: Passive and Active White-box Inference Attacks against Centralized and Federated Learning
- Understanding Neural Networks Through Deep Visualization
- DARTS: Differentiable Architecture Search
- Sensitivity and Generalization in Neural Networks: an Empirical Study
- SVCCA: Singular Vector Canonical Correlation Analysis for Deep Learning Dynamics and Interpretability
- Deep Models Under the GAN: Information Leakage from Collaborative Deep Learning
- Do Convolutional Neural Networks Learn Class Hierarchy?
- Topology of Learning in Artificial Neural Networks
- Stuck in a What? Adventures in Weight Space
- PCA of high dimensional random walks with comparison to neural network training