Exact distribution of the output of a deep-layered machine

Abstract

Deep-layered machines, in which each node computes a Boolean function of all nodes below it, underpin deep learning and digital computation. Yet the statistics of their global output function remain poorly understood. We derive the exact finite-depth distribution of the output of a machine with width $k$ and depth $n$. The distribution depends only on the Hamming weight of the output, and as $n$ increases favors functions with low and high Hamming weights. But this bias peaks at a crossover depth proportional to $2^k$ before collapsing onto the constant functions true and false.