Neural Persistence: A Complexity Measure for Deep Neural Networks Using Algebraic Topology

Abstract

While many approaches to make neural networks more fathomable have been proposed, they are restricted to interrogating the network with input data. Measures for characterizing and monitoring structural properties, however, have not been developed. In this work, we propose neural persistence, a complexity measure for neural network architectures based on topological data analysis on weighted stratified graphs. To demonstrate the usefulness of our approach, we show that neural persistence reflects best practices developed in the deep learning community such as dropout and batch normalization. Moreover, we derive a neural persistence-based stopping criterion that shortens the training process while achieving comparable accuracies as early stopping based on validation loss.

arXiv commentPublished as a conference paper at ICLR 2019

References in corpus (11)

• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• Sequence to Sequence Learning with Neural Networks
• Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms
• Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation
• Methods for Interpreting and Understanding Deep Neural Networks
• CheXNet: Radiologist-Level Pneumonia Detection on Chest X-Rays with Deep Learning
• Opening the Black Box of Deep Neural Networks via Information
• Statistical topological data analysis using persistence landscapes
• Persistent Homology of Complex Networks
• Geometry Score: A Method For Comparing Generative Adversarial Networks
• On Characterizing the Capacity of Neural Networks using Algebraic Topology

Cited by in corpus (20)

• A Topological Loss Function for Deep-Learning based Image Segmentation using Persistent Homology
• Simplicial Neural Networks
• A Topology Layer for Machine Learning
• Uncovering the Topology of Time-Varying fMRI Data using Cubical Persistence
• Artificial Text Detection via Examining the Topology of Attention Maps
• Optimizing persistent homology based functions
• Path homologies of deep feedforward networks
• Activation Landscapes as a Topological Summary of Neural Network Performance
• Evaluating the Disentanglement of Deep Generative Models through Manifold Topology
• PLLay: Efficient Topological Layer based on Persistence Landscapes
• Fuzzy c-Means Clustering for Persistence Diagrams
• Intrinsic Dimension, Persistent Homology and Generalization in Neural Networks
• RicciNets: Curvature-guided Pruning of High-performance Neural Networks Using Ricci Flow
• Topological Uncertainty: Monitoring trained neural networks through persistence of activation graphs
• A Fast and Robust Method for Global Topological Functional Optimization
• Finding the Homology of Decision Boundaries with Active Learning
• Graph Filtration Learning
• Periodic Spectral Ergodicity: A Complexity Measure for Deep Neural Networks and Neural Architecture Search
• Characterizing and Measuring the Similarity of Neural Networks with Persistent Homology
• Topologically Densified Distributions