Activation Landscapes as a Topological Summary of Neural Network Performance

Abstract

We use topological data analysis (TDA) to study how data transforms as it passes through successive layers of a deep neural network (DNN). We compute the persistent homology of the activation data for each layer of the network and summarize this information using persistence landscapes. The resulting feature map provides both an informative visual- ization of the network and a kernel for statistical analysis and machine learning. We observe that the topological complexity often increases with training and that the topological complexity does not decrease with each layer.

arXiv comment4 pages, 5 figures

References in corpus (16)

• Adam: A Method for Stochastic Optimization
• PyTorch: An Imperative Style, High-Performance Deep Learning Library
• Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization
• Understanding Neural Networks Through Deep Visualization
• Statistical topological data analysis using persistence landscapes
• Ripser: efficient computation of Vietoris-Rips persistence barcodes
• Enhanced Higgs to ${\tau }^{+}{\tau }^{-}$ Searches with Deep Learning
• PersLay: A Neural Network Layer for Persistence Diagrams and New Graph Topological Signatures
• Neural Persistence: A Complexity Measure for Deep Neural Networks Using Algebraic Topology
• On Characterizing the Capacity of Neural Networks using Algebraic Topology
• Topological Data Analysis for Arrhythmia Detection through Modular Neural Networks
• Connectivity-Optimized Representation Learning via Persistent Homology
• Applying Topological Persistence in Convolutional Neural Network for Music Audio Signals
• Local cohomology and stratification
• Evaluating the Disentanglement of Deep Generative Models through Manifold Topology
• Local homology of abstract simplicial complexes

Cited by in corpus (1)

• Topology-Aware Activation Functions in Neural Networks