832 citations · 1.3k across the 79 of their papers we have counts for
34 papers · 1 filter
Polyhedral Geometry of Time-to-First-Spike Neural Networks
Manjot Singh, Guido Montúfar, Gitta Kutyniok
We study the expressivity of spiking neural networks, which provide a natural framework for asynchronous, event-driven computation complementary to conventional feedforward neural…
TriSearch: Learning to Optimize Triangulations via Bistellar Flips
Yiran Wang, Guido Montúfar
We introduce TriSearch, a reinforcement learning framework for optimizing objectives over triangulations of a polytope via bistellar flips. The key idea is a circuit-supported subt…
Implicit Bias of Mirror Flow in Homogeneous Neural Networks: Sparse and Dense Feature Learning
Tom Jacobs, Guido Montufar
We study the max-margin solutions reached by mirror flow in deep neural networks with homogeneous activation functions. Extending classical results on gradient flow, we derive a no…
Most ReLU Networks Admit Identifiable Parameters
Moritz Grillo, Guido Montúfar
We study the realization map of deep ReLU networks, focusing on when a function determines its parameters up to scaling and permutation. To analyze hidden redundancies beyond these…
The Symmetries of Three-Layer ReLU Networks
Johanna Marie Gegenfurtner, Moritz Grillo, Guido Montúfar
We develop a framework for analyzing parameter symmetries in deep ReLU networks and obtain a complete characterization of the generic parameter fibers for three-layer bottleneck ar…
Stress-Testing Neural Network Verifiers with Provably Robust Instances
David Troxell, Yulia Alexandr, Sofia Hunt +2
Neural network verifiers aim to provide formal guarantees on model behavior, but existing verification benchmarks are fundamentally limited by their lack of ground-truth labels. As…
Differentiable Optimization Layers for Guaranteed Fairness in Deep Learning
David Troxell, Noah Roemer, Guido Montúfar
Differentiable optimization layers are traditionally integrated in predict-then-optimize frameworks where a neural model estimates parameters that subsequently serve as fixed input…
Low Rank Gradients and Where to Find Them
Rishi Sonthalia, Michael Murray, Guido Montúfar
This paper investigates low-rank structure in the gradients of the training loss for two-layer neural networks while relaxing the usual isotropy assumptions on the training data an…
Gradient Descent with Large Step Sizes: Chaos and Fractal Convergence Region
Shuang Liang, Guido Montúfar
We examine gradient descent in matrix factorization and show that under large step sizes the parameter space develops a fractal structure. We derive the exact critical step size fo…
Zero-Shot Context Generalization in Reinforcement Learning from Few Training Contexts
James Chapman, Kedar Karhadkar, Guido Montufar
Deep reinforcement learning (DRL) has achieved remarkable success across multiple domains, including competitive games, natural language processing, and robotics. Despite these adv…
Demystifying Topological Message-Passing with Relational Structures: A Case Study on Oversquashing in Simplicial Message-Passing
Diaaeldin Taha, James Chapman, Marzieh Eidi +2
Topological deep learning (TDL) has emerged as a powerful tool for modeling higher-order interactions in relational data. However, phenomena such as oversquashing in topological me…
On the Local Complexity of Linear Regions in Deep ReLU Networks
Niket Patel, Guido Montufar
We define the local complexity of a neural network with continuous piecewise linear activations as a measure of the density of linear regions over an input data distribution. We sh…
Mildly Overparameterized ReLU Networks Have a Favorable Loss Landscape
Kedar Karhadkar, Michael Murray, Hanna Tseran +1
We study the loss landscape of both shallow and deep, mildly overparameterized ReLU neural networks on a generic finite input dataset for the squared error loss. We show both by co…
Function Space and Critical Points of Linear Convolutional Networks
Kathlén Kohn, Guido Montúfar, Vahid Shahverdi +1
We study the geometry of linear networks with one-dimensional convolutional layers. The function spaces of these networks can be identified with semi-algebraic families of polynomi…
Characterizing the Spectrum of the NTK via a Power Series Expansion
Michael Murray, Hui Jin, Benjamin Bowman +1
Under mild conditions on the network initialization we derive a power series expansion for the Neural Tangent Kernel (NTK) of arbitrarily deep feedforward networks in the infinite…
FoSR: First-order spectral rewiring for addressing oversquashing in GNNs
Kedar Karhadkar, Pradeep Kr. Banerjee, Guido Montúfar
Graph neural networks (GNNs) are able to leverage the structure of graph data by passing messages along the edges of the graph. While this allows GNNs to learn features depending o…
Oversquashing in GNNs through the lens of information contraction and graph expansion
Pradeep Kr. Banerjee, Kedar Karhadkar, Yu Guang Wang +2
The quality of signal propagation in message-passing graph neural networks (GNNs) strongly influences their expressivity as has been observed in recent works. In particular, for pr…
Solving infinite-horizon POMDPs with memoryless stochastic policies in state-action space
Johannes Müller, Guido Montúfar
Reward optimization in fully observable Markov decision processes is equivalent to a linear program over the polytope of state-action frequencies. Taking a similar perspective in t…
Learning curves for Gaussian process regression with power-law priors and targets
Hui Jin, Pradeep Kr. Banerjee, Guido Montúfar
We characterize the power-law asymptotics of learning curves for Gaussian process regression (GPR) under the assumption that the eigenspectrum of the prior and the eigenexpansion c…
Training Wasserstein GANs without gradient penalties
Dohyun Kwon, Yeoneung Kim, Guido Montúfar +1
We propose a stable method to train Wasserstein generative adversarial networks. In order to enhance stability, we consider two objective functions using the -transform based on…
Geometry of Linear Convolutional Networks
Kathlén Kohn, Thomas Merkh, Guido Montúfar +1
We study the family of functions that are represented by a linear convolutional neural network (LCN). These functions form a semi-algebraic subset of the set of linear maps from in…
Weisfeiler and Lehman Go Cellular: CW Networks
Cristian Bodnar, Fabrizio Frasca, Nina Otter +4
Graph Neural Networks (GNNs) are limited in their expressive power, struggle with long-range interactions and lack a principled way to model higher-order structures. These problems…
Information Complexity and Generalization Bounds
Pradeep Kr. Banerjee, Guido Montúfar
We present a unifying picture of PAC-Bayesian and mutual information-based upper bounds on the generalization error of randomized learning algorithms. As we show, Tong Zhang's info…
Weisfeiler and Lehman Go Topological: Message Passing Simplicial Networks
Cristian Bodnar, Fabrizio Frasca, Yu Guang Wang +4
The pairwise interaction paradigm of graph machine learning has predominantly governed the modelling of relational systems. However, graphs alone cannot capture the multi-level int…