61 citations · 297 across the 23 of their papers we have counts for
4 papers · 1 filter
Using alternating de Bruijn sequences to construct de Bruijn tori
Matthew Kreitzer, Mihai Nica, Rajesh Pereira
A de Bruijn torus is the two dimensional generalization of a de Bruijn sequence. While some methods exist to generate these tori, only a few methods of construction are known. We p…
Network Degeneracy as an Indicator of Training Performance: Comparing Finite and Infinite Width Angle Predictions
Cameron Jakub, Mihai Nica
Neural networks are powerful functions with widespread use, but the theoretical behaviour of these functions is not fully understood. Creating deep neural networks by stacking many…
Dynamic Sparse Training with Structured Sparsity
Mike Lasby, Anna Golubeva, Utku Evci +2
Dynamic Sparse Training (DST) methods achieve state-of-the-art results in sparse neural network training, matching the generalization of dense models while enabling sparse training…
Depth Degeneracy in Neural Networks: Vanishing Angles in Fully Connected ReLU Networks on Initialization
Cameron Jakub, Mihai Nica
Despite remarkable performance on a variety of tasks, many properties of deep neural networks are not yet theoretically understood. One such mystery is the depth degeneracy phenome…