Network Horizon Dynamics I: Qualitative Aspects

Abstract

Mostly acyclic directed networks, treated mathematically as directed graphs, arise in machine learning, biology, social science, physics, and other applications. Newman [1] has noted the mathematical challenges of such networks. In this series of papers, we study their connectivity properties, focusing on three types of phase transitions that affect horizon sizes for typical nodes. The first two types involve the familiar emergence of giant components as average local connectivity increases, while the third type involves small-world horizon growth at variable distance from a typical node. In this first paper, we focus on qualitative behavior, simulations, and applications, leaving formal considerations for subsequent papers. We explain how such phase transitions distinguish deep neural networks from shallow machine learning architectures, and propose hybrid local/random network designs with surprising connectivity advantages. We also propose a small-world approach to the horizon problem in the cosmology of the early universe as a novel alternative to the inflationary hypothesis of Guth and Linde.

arXiv comment45 pages, 22 figures

References in corpus (21)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Statistical mechanics of complex networks
• Deep Learning in Neural Networks: An Overview
• Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling
• Random graphs with arbitrary degree distributions and their applications
• Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm
• Quantum algorithms for supervised and unsupervised machine learning
• Layer Normalization
• BICEP2 / Keck Array x: Constraints on Primordial Gravitational Waves using Planck, WMAP, and New BICEP2/Keck Observations through the 2015 Season
• An Empirical Evaluation of Deep Learning on Highway Driving
• Superluminary Universe: A Possible Solution to the Initial Value Problem in Cosmology
• Inflationary paradigm in trouble after Planck2013
• A Classical Sequential Growth Dynamics for Causal Sets
• Giant strongly connected component of directed networks
• Optical clock comparison test of Lorentz symmetry
• An exact mapping between the Variational Renormalization Group and Deep Learning
• Is the Universe homogeneous?
• Second- and First-Order Phase Transitions in CDT
• A Lorentzian cure for Euclidean troubles
• A closed form expression for the causal set d'Alembertian
• On the Axioms of Causal Set Theory