10 papers
Singular Learning and Occam's Razor in Deep Monomial Networks
Kathlén Kohn, Giovanni Luca Marchetti, Farhan Shabir +2
In the optimization of neural networks, gradient dynamics are influenced by critical points that arise from the model's architecture. These critical points occur where the Jacobian…
Conservation Laws from Data Symmetry in Neural Networks
Jakob Galley, Vahid Shahverdi, Axel Flinth
We explore whether intrinsic symmetries of the training data lead to conserved quantities during gradient-flow training of neural networks. Under the assumption that the loss funct…
Identifiable Equivariant Networks are Layerwise Equivariant
Vahid Shahverdi, Giovanni Luca Marchetti, Georg Bökman +1
We investigate the relation between end-to-end equivariance and layerwise equivariance in deep neural networks. We prove the following: For a network whose end-to-end function is e…
Learning on a Razor's Edge: Identifiability and Singularity of Polynomial Neural Networks
Vahid Shahverdi, Giovanni Luca Marchetti, Kathlén Kohn
We study function spaces parametrized by neural networks, referred to as neuromanifolds. Specifically, we focus on deep Multi-Layer Perceptrons (MLPs) and Convolutional Neural Netw…
Algebra Unveils Deep Learning -- An Invitation to Neuroalgebraic Geometry
Giovanni Luca Marchetti, Vahid Shahverdi, Stefano Mereta +2
In this position paper, we promote the study of function spaces parameterized by machine learning models through the lens of algebraic geometry. To this end, we focus on algebraic…
On the Geometry and Optimization of Polynomial Convolutional Networks
Vahid Shahverdi, Giovanni Luca Marchetti, Kathlén Kohn
We study convolutional neural networks with monomial activation functions. Specifically, we prove that their parameterization map is regular and is an isomorphism almost everywhere…
Moment Constraints and Phase Recovery for Multireference Alignment
Vahid Shahverdi, Emanuel Ström, Joakim Andén
Multireference alignment (MRA) refers to the problem of recovering a signal from noisy samples subject to random circular shifts. Expectation--maximization (EM) and variational app…
Algebraic Complexity and Neurovariety of Linear Convolutional Networks
Vahid Shahverdi
In this paper, we study linear convolutional networks with one-dimensional filters and arbitrary strides. The neuromanifold of such a network is a semialgebraic set, represented by…
Geometry of Linear Neural Networks: Equivariance and Invariance under Permutation Groups
Kathlén Kohn, Anna-Laura Sattelberger, Vahid Shahverdi
The set of functions parameterized by a linear fully-connected neural network is a determinantal variety. We investigate the subvariety of functions that are equivariant or invaria…
Function Space and Critical Points of Linear Convolutional Networks
Kathlén Kohn, Guido Montúfar, Vahid Shahverdi +1
We study the geometry of linear networks with one-dimensional convolutional layers. The function spaces of these networks can be identified with semi-algebraic families of polynomi…