3 citations · 15 across the 27 of their papers we have counts for
17 papers · 1 filter
Singular Learning and Occam's Razor in Deep Monomial Networks
Kathlén Kohn, Giovanni Luca Marchetti, Farhan Shabir +2
In the optimization of neural networks, gradient dynamics are influenced by critical points that arise from the model's architecture. These critical points occur where the Jacobian…
Sequential Group Composition: A Window into the Mechanics of Deep Learning
Giovanni Luca Marchetti, Daniel Kunin, Adele Myers +2
How do neural networks trained over sequences acquire the ability to perform structured operations, such as arithmetic, geometric, and algorithmic computation? To gain insight into…
Identifiable Equivariant Networks are Layerwise Equivariant
Vahid Shahverdi, Giovanni Luca Marchetti, Georg Bökman +1
We investigate the relation between end-to-end equivariance and layerwise equivariance in deep neural networks. We prove the following: For a network whose end-to-end function is e…
A Non-Adversarial Approach to Idempotent Generative Modelling
Mohammed Al-Jaff, Giovanni Luca Marchetti, Michael C Welle +5
Idempotent Generative Networks (IGNs) are deep generative models that also function as local data manifold projectors, mapping arbitrary inputs back onto the manifold. They are tra…
Alternating Gradient Flows: A Theory of Feature Learning in Two-layer Neural Networks
Daniel Kunin, Giovanni Luca Marchetti, Feng Chen +5
What features neural networks learn, and how, remains an open question. In this paper, we introduce Alternating Gradient Flows (AGF), an algorithmic framework that describes the dy…
Learning on a Razor's Edge: Identifiability and Singularity of Polynomial Neural Networks
Vahid Shahverdi, Giovanni Luca Marchetti, Kathlén Kohn
We study function spaces parametrized by neural networks, referred to as neuromanifolds. Specifically, we focus on deep Multi-Layer Perceptrons (MLPs) and Convolutional Neural Netw…
Algebra Unveils Deep Learning -- An Invitation to Neuroalgebraic Geometry
Giovanni Luca Marchetti, Vahid Shahverdi, Stefano Mereta +2
In this position paper, we promote the study of function spaces parameterized by machine learning models through the lens of algebraic geometry. To this end, we focus on algebraic…
On the Geometry and Optimization of Polynomial Convolutional Networks
Vahid Shahverdi, Giovanni Luca Marchetti, Kathlén Kohn
We study convolutional neural networks with monomial activation functions. Specifically, we prove that their parameterization map is regular and is an isomorphism almost everywhere…
Relative Representations: Topological and Geometric Perspectives
Alejandro García-Castellanos, Giovanni Luca Marchetti, Danica Kragic +1
Relative representations are an established approach to zero-shot model stitching, consisting of a non-trainable transformation of the latent space of a deep neural network. Based…
Geometry of Lightning Self-Attention: Identifiability and Dimension
Nathan W. Henry, Giovanni Luca Marchetti, Kathlén Kohn
We consider function spaces defined by self-attention networks without normalization, and theoretically analyze their geometry. Since these networks are polynomial, we rely on tool…
Hyperbolic Delaunay Geometric Alignment
Aniss Aiman Medbouhi, Giovanni Luca Marchetti, Vladislav Polianskii +4
Hyperbolic machine learning is an emerging field aimed at representing data with a hierarchical structure. However, there is a lack of tools for evaluation and analysis of the resu…
Harmonics of Learning: Universal Fourier Features Emerge in Invariant Networks
Giovanni Luca Marchetti, Christopher Hillar, Danica Kragic +1
In this work, we formally prove that, under certain conditions, if a neural network is invariant to a finite group then its weights recover the Fourier transform on that group. Thi…
Neural Lattice Reduction: A Self-Supervised Geometric Deep Learning Approach
Giovanni Luca Marchetti, Gabriele Cesa, Pratik Kumar +1
Lattice reduction is a combinatorial optimization problem aimed at finding the most orthogonal basis in a given lattice. The Lenstra-Lenstra-Lovász (LLL) algorithm is the best algo…
Learning Geometric Representations of Objects via Interaction
Alfredo Reichlin, Giovanni Luca Marchetti, Hang Yin +2
We address the problem of learning representations from observations of a scene involving an agent and an external object the agent interacts with. To this end, we propose a repres…
Back to the Manifold: Recovering from Out-of-Distribution States
Alfredo Reichlin, Giovanni Luca Marchetti, Hang Yin +2
Learning from previously collected datasets of expert data offers the promise of acquiring robotic policies without unsafe and costly online explorations. However, a major challeng…
Equivariant Representation Learning via Class-Pose Decomposition
Giovanni Luca Marchetti, Gustaf Tegnér, Anastasiia Varava +1
We introduce a general method for learning representations that are equivariant to symmetries of data. Our central idea is to decompose the latent space into an invariant factor an…
Active Nearest Neighbor Regression Through Delaunay Refinement
Alexander Kravberg, Giovanni Luca Marchetti, Vladislav Polianskii +3
We introduce an algorithm for active function approximation based on nearest neighbor regression. Our Active Nearest Neighbor Regressor (ANNR) relies on the Voronoi-Delaunay framew…