465 citations · 1.5k across the 12 of their papers we have counts for
3 papers · 1 filter
Two Sparsities Are Better Than One: Unlocking the Performance Benefits of Sparse-Sparse Networks
Kevin Lee Hunter, Lawrence Spracklen, Subutai Ahmad
In principle, sparse neural networks should be significantly more efficient than traditional dense networks. Neurons in the brain exhibit two types of sparsity; they are sparsely i…
Long Distance Relationships without Time Travel: Boosting the Performance of a Sparse Predictive Autoencoder in Sequence Modeling
Jeremy Gordon, David Rawlinson, Subutai Ahmad
In sequence learning tasks such as language modelling, Recurrent Neural Networks must learn relationships between input features separated by time. State of the art models such as…
How Can We Be So Dense? The Benefits of Using Highly Sparse Representations
Subutai Ahmad, Luiz Scheinkman
Most artificial networks today rely on dense representations, whereas biological networks rely on sparse representations. In this paper we show how sparse representations can be mo…