Topos and Stacks of Deep Neural Networks
arXiv:2106.14587
Abstract
Every known artificial deep neural network (DNN) corresponds to an object in a canonical Grothendieck's topos; its learning dynamic corresponds to a flow of morphisms in this topos. Invariance structures in the layers (like CNNs or LSTMs) correspond to Giraud's stacks. This invariance is supposed to be responsible of the generalization property, that is extrapolation from learning data under constraints. The fibers represent pre-semantic categories (Culioli, Thom), over which artificial languages are defined, with internal logics, intuitionist, classical or linear (Girard). Semantic functioning of a network is its ability to express theories in such a language for answering questions in output about input data. Quantities and spaces of semantic information are defined by analogy with the homological interpretation of Shannon's entropy of P.Baudot and D.Bennequin in 2015). They generalize the measures found by Carnap and Bar-Hillel (1952). Amazingly, the above semantical structures are classified by geometric fibrant objects in a closed model category of Quillen, then they give rise to homotopical invariants of DNNs and of their semantic functioning. Intentional type theories (Martin-Loef) organize these objects and fibrations between them. Information contents and exchanges are analyzed by Grothendieck's derivators.
151 pages, 12 figures
References in corpus (13)
- Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling
- Deep Learning Enabled Semantic Communication Systems
- A simple neural network module for relational reasoning
- Gauge Equivariant Convolutional Networks and the Icosahedral CNN
- Deep Visual-Semantic Alignments for Generating Image Descriptions
- Discovering objects and their relations from entangled scene representations
- Exact completions and small sheaves
- Homotopy Theoretic Models of Type Theory
- Simplified Minimal Gated Unit Variations for Recurrent Neural Networks
- Homotopy Theoretic and Categorical Models of Neural Information Networks
- The Hintons in your Neural Network: a Quantum Field Theory View of Deep Learning
- Pursuing Stacks
- Interaction decomposition for presheaves