832 citations · 1.3k across the 79 of their papers we have counts for
7 papers · 1 filter
Hadamard ranks of algebraic varieties
Dario Antolini, Guido Montúfar, Alessandro Oneto
Motivated by the study of decompositions of tensors as Hadamard products (i.e., coefficient-wise products) of low-rank tensors, we introduce the notion of Hadamard rank of a given…
Low Rank Gradients and Where to Find Them
Rishi Sonthalia, Michael Murray, Guido Montúfar
This paper investigates low-rank structure in the gradients of the training loss for two-layer neural networks while relaxing the usual isotropy assumptions on the training data an…
Gradient Descent with Large Step Sizes: Chaos and Fractal Convergence Region
Shuang Liang, Guido Montúfar
We examine gradient descent in matrix factorization and show that under large step sizes the parameter space develops a fractal structure. We derive the exact critical step size fo…
Constraining the outputs of ReLU neural networks
Yulia Alexandr, Guido Montúfar
We introduce a class of algebraic varieties naturally associated with ReLU neural networks, arising from the piecewise linear structure of their outputs across activation regions i…
Zero-Shot Context Generalization in Reinforcement Learning from Few Training Contexts
James Chapman, Kedar Karhadkar, Guido Montufar
Deep reinforcement learning (DRL) has achieved remarkable success across multiple domains, including competitive games, natural language processing, and robotics. Despite these adv…
Understanding Learning Invariance in Deep Linear Networks
Hao Duan, Guido Montúfar
Equivariant and invariant machine learning models exploit symmetries and structural patterns in data to improve sample efficiency. While empirical studies suggest that data-driven…
Demystifying Topological Message-Passing with Relational Structures: A Case Study on Oversquashing in Simplicial Message-Passing
Diaaeldin Taha, James Chapman, Marzieh Eidi +2
Topological deep learning (TDL) has emerged as a powerful tool for modeling higher-order interactions in relational data. However, phenomena such as oversquashing in topological me…