Network Horizon Dynamics I: Qualitative Aspects

Abstract

Mostly acyclic directed networks, treated mathematically as directed graphs, arise in machine learning, biology, social science, physics, and other applications. Newman [1] has noted the mathematical challenges of such networks. In this series of papers, we study their connectivity properties, focusing on three types of phase transitions that affect horizon sizes for typical nodes. The first two types involve the familiar emergence of giant components as average local connectivity increases, while the third type involves small-world horizon growth at variable distance from a typical node. In this first paper, we focus on qualitative behavior, simulations, and applications, leaving formal considerations for subsequent papers. We explain how such phase transitions distinguish deep neural networks from shallow machine learning architectures, and propose hybrid local/random network designs with surprising connectivity advantages. We also propose a small-world approach to the horizon problem in the cosmology of the early universe as a novel alternative to the inflationary hypothesis of Guth and Linde.

45 pages, 22 figures

References in corpus (8)

• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Deep Learning in Neural Networks: An Overview
• Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling
• Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm
• Layer Normalization
• An Empirical Evaluation of Deep Learning on Highway Driving
• An exact mapping between the Variational Renormalization Group and Deep Learning
• Second- and First-Order Phase Transitions in CDT