9 papers
Removing spurious minima for planar features by skip connections
Jakob Paul Zimmermann, Moritz Grillo, Andrei Balakin +1
Understanding loss landscapes is central to explaining neural-network training, yet their structure remains only partially understood even in simple models. We study the Gaussian p…
Shallower ReLU Network Representations via Exact Linear Algebra
Kilian Rueß, Gennadiy Averkov, Florestan Brunck +7
We study the depth required by ReLU networks to exactly represent piecewise linear functions, focusing specifically on the maximum function. This problem has recently received sign…
Parameterized Complexity of -Lipschitz Constants for Input Convex Neural Networks and -Norm Maximization over Zonotopes
Aritra Das, Vincent Froese, Moritz Grillo +6
Lipschitz constants are a standard way to quantify the sensitivity of neural networks to small input perturbations, but computing them is difficult even for shallow ReLU networks.…
Most ReLU Networks Admit Identifiable Parameters
Moritz Grillo, Guido Montúfar
We study the realization map of deep ReLU networks, focusing on when a function determines its parameters up to scaling and permutation. To analyze hidden redundancies beyond these…
The Symmetries of Three-Layer ReLU Networks
Johanna Marie Gegenfurtner, Moritz Grillo, Guido Montúfar
We develop a framework for analyzing parameter symmetries in deep ReLU networks and obtain a complete characterization of the generic parameter fibers for three-layer bottleneck ar…
On the expressivity of sparse maxout networks
Moritz Grillo, Tobias Hofmann
We study the expressivity of sparse maxout networks, where each neuron takes a fixed number of inputs from the previous layer and employs a, possibly multi-argument, maxout activat…
Parameterized Hardness of Zonotope Containment and Neural Network Verification
Vincent Froese, Moritz Grillo, Christoph Hertrich +1
Neural networks with ReLU activations are a widely used model in machine learning. It is thus important to have a profound understanding of the properties of the functions computed…
Depth-Bounds for Neural Networks via the Braid Arrangement
Moritz Grillo, Christoph Hertrich, Georg Loho
We contribute towards resolving the open question of how many hidden layers are required in ReLU networks for exactly representing all continuous and piecewise linear functions on…
Decomposition Polyhedra of Piecewise Linear Functions
Marie-Charlotte Brandenburg, Moritz Grillo, Christoph Hertrich
In this paper we contribute to the frequently studied question of how to decompose a continuous piecewise linear (CPWL) function into a difference of two convex CPWL functions. Eve…