832 citations · 1.3k across the 79 of their papers we have counts for
5 papers · 1 filter
Supermodular Rank: Set Function Decomposition and Optimization
Rishi Sonthalia, Anna Seigal, Guido Montufar
We define the supermodular rank of a function on a lattice. This is the smallest number of terms needed to decompose it into a sum of supermodular functions. The supermodular summa…
Mildly Overparameterized ReLU Networks Have a Favorable Loss Landscape
Kedar Karhadkar, Michael Murray, Hanna Tseran +1
We study the loss landscape of both shallow and deep, mildly overparameterized ReLU neural networks on a generic finite input dataset for the squared error loss. We show both by co…
Function Space and Critical Points of Linear Convolutional Networks
Kathlén Kohn, Guido Montúfar, Vahid Shahverdi +1
We study the geometry of linear networks with one-dimensional convolutional layers. The function spaces of these networks can be identified with semi-algebraic families of polynomi…
Critical Points and Convergence Analysis of Generative Deep Linear Networks Trained with Bures-Wasserstein Loss
Pierre Bréchet, Katerina Papagiannouli, Jing An +1
We consider a deep matrix factorization model of covariance matrices trained with the Bures-Wasserstein distance. While recent works have made advances in the study of the optimiza…
Expected Gradients of Maxout Networks and Consequences to Parameter Initialization
Hanna Tseran, Guido Montúfar
We study the gradients of a maxout network with respect to inputs and parameters and obtain bounds for the moments depending on the architecture and the parameter distribution. We…