Minimal Filling Architectures for Polynomial Neural Networks

Abstract

Minimal filling architectures describe the smallest hidden-layer widths for which the neurovariety of a polynomial neural network fills its ambient function space. It was conjectured that every such architecture has unimodal hidden widths. We construct counterexamples to this conjecture using a frontier search and certify the main example through finite-field Jacobian computations and recursive dimension bounds. In the opposite direction, we prove that every minimal filling architecture of depth at most three is unimodal. A nonunimodal minimal filling architecture of depth four shows that the depth bound is sharp.

arXiv comment26 pages, 3 figures, 3 tables. Added results on low depth architectures, more counterexamples, rewritten appendix, and updates to exposition