SVCCA: Singular Vector Canonical Correlation Analysis for Deep Learning Dynamics and Interpretability

Abstract

We propose a new technique, Singular Vector Canonical Correlation Analysis (SVCCA), a tool for quickly comparing two representations in a way that is both invariant to affine transform (allowing comparison between different layers and networks) and fast to compute (allowing more comparisons to be calculated than with previous methods). We deploy this tool to measure the intrinsic dimensionality of layers, showing in some cases needless over-parameterization; to probe learning dynamics throughout training, finding that networks converge to final representations from the bottom up; to show where class-specific information in networks is formed; and to suggest new training regimes that simultaneously save computation and overfit less. Code: https://github.com/google/svcca/

Accepted to NIPS 2017, code: https://github.com/google/svcca/ , new plots on Imagenet

Cited by in corpus (90)

• A Survey on Explainable Artificial Intelligence (XAI): Towards Medical XAI
• Transfusion: Understanding Transfer Learning for Medical Imaging
• Multi-task Self-Supervised Learning for Human Activity Detection
• High-Performance Large-Scale Image Recognition Without Normalization
• Intrinsic dimension of data representations in deep neural networks
• Do Wide and Deep Networks Learn the Same Things? Uncovering How Neural Network Representations Vary with Width and Depth
• Rapid Learning or Feature Reuse? Towards Understanding the Effectiveness of MAML
• Explainable Artificial Intelligence: a Systematic Review
• A Survey of Deep Learning for Scientific Discovery
• ModelDiff: Testing-Based DNN Similarity Comparison for Model Reuse Detection
• Universality and individuality in neural dynamics across large populations of recurrent networks
• Neural network interpretation using descrambler groups
• Emerging Cross-lingual Structure in Pretrained Language Models
• The Shape of Data: Intrinsic Distance for Data Distributions
• A Comprehensive Survey of Multilingual Neural Machine Translation
• Anatomy of Catastrophic Forgetting: Hidden Representations and Task Semantics
• Revisiting Model Stitching to Compare Neural Representations
• Deep Learning Through the Lens of Example Difficulty
• Selectivity considered harmful: evaluating the causal impact of class selectivity in DNNs
• A Gradient Flow Framework For Analyzing Network Pruning
• The Bottom-up Evolution of Representations in the Transformer: A Study with Machine Translation and Language Modeling Objectives
• Adversarial training in communication constrained federated learning
• On Linear Identifiability of Learned Representations
• A Study of Gradient Variance in Deep Learning
• Similarity of Neural Networks with Gradients
• Topic Modeling with Contextualized Word Representation Clusters
• The Life Cycle of Knowledge in Big Language Models: A Survey
• MURAL: Multimodal, Multitask Retrieval Across Languages
• On the geometry of generalization and memorization in deep neural networks
• An Interpretability Illusion for BERT
• Improving Zero-Shot Translation by Disentangling Positional Information
• Shared Representational Geometry Across Neural Networks
• Interference and Generalization in Temporal Difference Learning
• Emergence of Separable Manifolds in Deep Language Representations
• Generative Feature Replay For Class-Incremental Learning
• COALA: Co-Aligned Autoencoders for Learning Semantically Enriched Audio Representations
• Investigating Multilingual NMT Representations at Scale
• PURSUhInT: In Search of Informative Hint Points Based on Layer Clustering for Knowledge Distillation
• Optimizing Mode Connectivity via Neuron Alignment
• Measuring Dependence with Matrix-based Entropy Functional
• On Negative Interference in Multilingual Models: Findings and A Meta-Learning Treatment
• Transferred Discrepancy: Quantifying the Difference Between Representations
• Classifying the classifier: dissecting the weight space of neural networks
• Exploring the Interchangeability of CNN Embedding Spaces
• On Geometric Structure of Activation Spaces in Neural Networks
• Let's Agree to Agree: Neural Networks Share Classification Order on Real Datasets
• Spectral Analysis of Latent Representations
• Simon Says: Evaluating and Mitigating Bias in Pruned Neural Networks with Knowledge Distillation
• Grounding Representation Similarity with Statistical Testing
• TX-Ray: Quantifying and Explaining Model-Knowledge Transfer in (Un-)Supervised NLP
• Internal representation dynamics and geometry in recurrent neural networks
• RePr: Improved Training of Convolutional Filters
• Walking the Tightrope: An Investigation of the Convolutional Autoencoder Bottleneck
• ACDC: Weight Sharing in Atom-Coefficient Decomposed Convolution
• Ptolemy: Architecture Support for Robust Deep Learning
• When and how epochwise double descent happens
• Improving Speech Translation by Understanding and Learning from the Auxiliary Text Translation Task
• Understanding Learning Dynamics for Neural Machine Translation
• Hierarchical nucleation in deep neural networks
• Recurrent Neural Network Learning of Performance and Intrinsic Population Dynamics from Sparse Neural Data
• Understanding the Effects of Pre-Training for Object Detectors via Eigenspectrum
• Improving Differentially Private Models with Active Learning
• Student Specialization in Deep ReLU Networks With Finite Width and Input Dimension
• Sampling for Deep Learning Model Diagnosis (Technical Report)
• Subspace Match Probably Does Not Accurately Assess the Similarity of Learned Representations
• Modeling Object Dissimilarity for Deep Saliency Prediction
• Properties of the After Kernel
• Experiments with Rich Regime Training for Deep Learning
• Similarity Analysis of Self-Supervised Speech Representations
• Analyzing Visual Representations in Embodied Navigation Tasks
• ImageNet Pre-training also Transfers Non-Robustness
• Comparison Against Task Driven Artificial Neural Networks Reveals Functional Organization of Mouse Visual Cortex
• Benchmarking CNN on 3D Anatomical Brain MRI: Architectures, Data Augmentation and Deep Ensemble Learning
• Low-Cost Transfer Learning of Face Tasks
• Layer Pruning on Demand with Intermediate CTC
• Coherence of Working Memory Study Between Deep Neural Network and Neurophysiology
• CACTUS: Detecting and Resolving Conflicts in Objective Functions
• Neural Supervised Domain Adaptation by Augmenting Pre-trained Models with Random Units
• One-Shot Neural Ensemble Architecture Search by Diversity-Guided Search Space Shrinking
• An Investigation of the Weight Space to Monitor the Training Progress of Neural Networks
• Analyzing and Interpreting Neural Networks for NLP: A Report on the First BlackboxNLP Workshop
• Visualizing Classification Structure of Large-Scale Classifiers
• Analyzing Representations inside Convolutional Neural Networks
• Understanding the Dynamics of DNNs Using Graph Modularity
• Verifiability and Predictability: Interpreting Utilities of Network Architectures for Point Cloud Processing
• Measuring the Stability of Learned Features
• Facial Landmark Correlation Analysis
• OrthoReg: Robust Network Pruning Using Orthonormality Regularization
• Training Efficiency and Robustness in Deep Learning
• Zero-shot Speech Translation