61 citations · 297 across the 23 of their papers we have counts for
6 papers · 1 filter
Network Degeneracy as an Indicator of Training Performance: Comparing Finite and Infinite Width Angle Predictions
Cameron Jakub, Mihai Nica
Neural networks are powerful functions with widespread use, but the theoretical behaviour of these functions is not fully understood. Creating deep neural networks by stacking many…
Dynamic Sparse Training with Structured Sparsity
Mike Lasby, Anna Golubeva, Utku Evci +2
Dynamic Sparse Training (DST) methods achieve state-of-the-art results in sparse neural network training, matching the generalization of dense models while enabling sparse training…
Bounding generalization error with input compression: An empirical study with infinite-width networks
Angus Galloway, Anna Golubeva, Mahmoud Salem +3
Estimating the Generalization Error (GE) of Deep Neural Networks (DNNs) is an important task that often relies on availability of held-out data. The ability to better predict GE ba…
The Exponentially Tilted Gaussian Prior for Variational Autoencoders
Griffin Floto, Stefan Kremer, Mihai Nica
An important property for deep neural networks is the ability to perform robust out-of-distribution detection on previously unseen data. This property is essential for safety purpo…
A Derivative-Free Method for Solving Elliptic Partial Differential Equations with Deep Neural Networks
Jihun Han, Mihai Nica, Adam R Stinchcombe
We introduce a deep neural network based method for solving a class of elliptic partial differential equations. We approximate the solution of the PDE with a deep neural network wh…
Finite Depth and Width Corrections to the Neural Tangent Kernel
Boris Hanin, Mihai Nica
We prove the precise scaling, at finite depth and width, for the mean and variance of the neural tangent kernel (NTK) in a randomly initialized ReLU network. The standard deviation…