5 papers · 1 filter
Shallower ReLU Network Representations via Exact Linear Algebra
Kilian RueÃ, Gennadiy Averkov, Florestan Brunck +7
We prove that the maximum of real numbers is exactly representable by a ReLU network with two hidden layers for every . The constructions are obtained by reducing the…
Most ReLU Networks Admit Identifiable Parameters
Moritz Grillo, Guido Montúfar
We study the realization map of deep ReLU networks, focusing on when a function determines its parameters up to scaling and permutation. To analyze hidden redundancies beyond these…
The Symmetries of Three-Layer ReLU Networks
Johanna Marie Gegenfurtner, Moritz Grillo, Guido Montúfar
We develop a framework for analyzing parameter symmetries in deep ReLU networks and obtain a complete characterization of the generic parameter fibers for three-layer bottleneck ar…
Depth-Bounds for Neural Networks via the Braid Arrangement
Moritz Grillo, Christoph Hertrich, Georg Loho
We contribute towards resolving the open question of how many hidden layers are required in ReLU networks for exactly representing all continuous and piecewise linear functions on…
On the expressivity of sparse maxout networks
Moritz Grillo, Tobias Hofmann
We study the expressivity of sparse maxout networks, where each neuron takes a fixed number of inputs from the previous layer and employs a, possibly multi-argument, maxout activat…