Path homologies of deep feedforward networks

Abstract

We provide a characterization of two types of directed homology for fully-connected, feedforward neural network architectures. These exact characterizations of the directed homology structure of a neural network architecture are the first of their kind. We show that the directed flag homology of deep networks reduces to computing the simplicial homology of the underlying undirected graph, which is explicitly given by Euler characteristic computations. We also show that the path homology of these networks is non-trivial in higher dimensions and depends on the number and size of the layers within the network. These results provide a foundation for investigating homological differences between neural network architectures and their realized structure as implied by their parameters.

arXiv commentTo appear in the proceedings of IEEE ICMLA 2019

References in corpus (16)

• Neural Architecture Search: A Survey
• The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks
• Pruning Filters for Efficient ConvNets
• Deep Learning with Topological Signatures
• Topology-Preserving Deep Image Segmentation
• Neural Persistence: A Complexity Measure for Deep Neural Networks Using Algebraic Topology
• On Characterizing the Capacity of Neural Networks using Algebraic Topology
• A Topological Regularizer for Classifiers via Persistent Homology
• A Topology Layer for Machine Learning
• Topology of Learning in Artificial Neural Networks
• Homologies of path complexes and digraphs
• Geometric Understanding of Deep Learning
• Connectivity-Optimized Representation Learning via Persistent Homology
• Rips filtrations for quasi-metric spaces and asymmetric functions with stability results
• Topological Approaches to Deep Learning
• Künneth Formulae in Persistent Homology

Cited by in corpus (7)

• Distances and Isomorphism between Networks: Stability and Convergence of Network Invariants
• Homotopy Theoretic and Categorical Models of Neural Information Networks
• Path homology and temporal networks
• Topology in cyber research
• Simplicial approach to path homology of quivers, marked categories, groups and algebras
• Path homology as a stronger analogue of cyclomatic complexity
• Characterizing and Measuring the Similarity of Neural Networks with Persistent Homology